Python: Combining 'complex' functions; one has oswalk other writes to file/database

Question

I have read numerous examples of combining functions in a main function, or calling them after both are defined, but all of the examples are very simple, and are not easily applied to what I am trying to do.

I will paste in the code so far. Please note that the second function here is shortened for the sake of making a quick print statement/checking if the files are being passed correctly by writing to an outfile. In the full second function, the value of 'n2' is actually intended to be entered into a SQlite3 database, such that I can identify which entries in the table are from a given file.

When I call these functions below individually, such as in the code below, I get a list of names from the first function, but

< function fastq2SQlite at 0x0000000011CD7208 >

for the second, and no file is created as specified in the function.

Thank you for your assistance.

import os
import fnmatch

def findFiles (path, filter):
    files = []
    for root, dirs, files in os.walk(path):
        for file in fnmatch.filter(files, filter):
            files.append(os.path.join(root, file))
    return files

def fastq2SQlite(files):
    nmlst = []
    out = ('path.txt', 'w')
    for f in files:
        names = f.split('\\')
        continue
    for name in names:
        n = name.split('.')
        n2 = n[0]
        nmlst.append(n2)
    out.write(n2)
    out.close()
    return n2

print findFiles('path', 'filter')
print fastq2SQlite()

Answer 1

You haven't passed files parameter into fastq2SQlite() function, try this:

print fastq2SQlite(findFiles('path', 'filter'))

And you'd probably thought after running your first function findFiles , the files will be automatically passed to second function, it is not working like this.

files in findFiles is a local namespace which isn't accessible from outside your function. And for your second function to work, you still need to pass the parameter files to it.

The strange behaviour you had was because you were trying to print a function without () , like this:

print fastq2SQlite
<function fastq2SQlite at 0x111917230>

And to actually call the function you need to add (whatever parameter you need to pass) , in your case i believe is this: fastq2SQlite(findFiles('path', 'filter')) , hope this helps.

Answer 2

One of your bugs is in this function:

def findFiles (path, filter):
    files = []
    for root, dirs, files in os.walk(path):
        for file in fnmatch.filter(files, filter):
            files.append(os.path.join(root, file))
    return files

You're using the identifier files twice: not just as the identifier for the list you're initializing to [] , appending to, and eventually returning, but also as a loop variable in the for root, dirs, files in os.walk(path): so it's being re-bound to a different list each time through the loop. Rename one of the two uses, eg to

def findFiles (path, filter):
    all_files = []
    for root, dirs, files in os.walk(path):
        for file in fnmatch.filter(files, filter):
            all_files.append(os.path.join(root, file))
    return all_files

to fix this one error. Another error is that in:

print findFiles('path', 'filter')
print fastq2SQlite()

you're just printing the list of files, not passing it on to fastq2SQlite . To fix this one, change the code eg to:

all_files = findFiles('path', 'filter')
print all_files
print fastq2SQlite(all_files)

Moreover, in fastq2SQlite , you assign out a tuple:

out = ('path.txt', 'w')

but then later you call out.write , which makes no sense. Maybe you meant

out = open('path.txt', 'w')

Further, in the for f in files: loop, the continue at the end of the loop is useless and redundant (though at least, differently from the other bugs, innocuous:-).

There may be other bugs yet, but even my "debugger eyes" are exhausted after finding three major ones in such a tiny compass...:-)