How do I iterate through folders to find particular files in Python?

Question

I am trying to iterate through folders to perform analysis on particular files.

Let's say I have 3 Folders named S1 , S2 , and S3 . Each folder contains a file with the same name, called hi.csv . Once I get the script to successfully iterate through each folder and find that file, I will have it analyze the CSV and create a new CSV file in each folder.

In lieu of the analysis code, I've just tried to print the files in the first instance. Ideally, I would see a file name called hi.txt under each subfolder, S1 , S2 and S3 .

Here is what I've tried so far, but it isn't working:

fn = 'hi.txt'
indir = '/Users/sheena/Desktop/Test'
for root, dirs, filenames in os.walk(indir):
    for d in dirs:
        if os.path.isfile(fn):
            print(f)

Answer 1

If you are trying to check for a certain filename, you should check it in filenames and if it is a success, the current folder is root . dirs doesn't enter the picture.

fn = 'hi.txt'
indir = '/Users/sheena/Desktop/Test'
for root, _, filenames in os.walk(indir):
    if fn in filenames:
        print("Got: {}/{}".format(root, fn))

See if it works the way you want. Please comment if it doesn't.

Answer 2

An easy way to see what's going wrong is to add a line

print(root, dirs, filenames)

just after the first 'for' loop. The output you'll get is something along the lines of

/tmp ['s3', 's2', 's1', ...] ['test.py', 'GRADUATE BALLOT APPLICATION FORM 2016-17.doc', 'ankid6955d9721560531274cb8f50ff595a9bd39d66f', '.X0-lock']
/tmp/s3 [] ['hi.txt']
/tmp/s2 [] ['hi.txt']
/tmp/s1 [] ['hi.txt']
/tmp/hsperfdata_joshua [] ['391']
/tmp/cujc ['foo'] []
/tmp/yaourt-tmp-joshua [] []
/tmp/.Test-unix [] []
/tmp/.font-unix [] []
/tmp/.XIM-unix [] []

So

1. "os.walk" may not be doing what you're expecting it to do, it's iterating over all subdirectories and subdirectories of subdirectories (but this isn't an issue for your code, there might just be a smarter/faster way of doing it)
2. All the files are in the "filenames" list - so you're iterating over the wrong list! 'hi.txt' is in the 'filenames' list when 'root' is 'blah/S1', 'blah/S2', or 'blah/S3'.

Answer 3

if os.path.isfile(os.path.join(d,fn)):

I guess ... maybe ... this probably is not the best way to find the files that match

for root, dirs, filenames in os.walk(indir):
    for fname in filenames:
        if fname == fn:
           print os.path.join(root,fname)

is probably a little better