Python os.walk，使用子目录中的文件

Question

Learning Python as part of my thesis, and I'm completely self taught/don't know what I'm doing; apologies if this is trivial. Been scouring the net for a while, and cannot find a "plain english" answer to this, nor can I figure out how to do this on my own. I've gleaned many tidbits on knowledge from this site via lurking, so I hope ya'll can help.

I have a folder that contains folders named after years (2001, 2002, 2003, etc), and inside each is a numbered folder for each month (01, 02, 03, etc). Inside each of those are my csv files that I'm working with. I've already set up a program to load them in and analyze them, but it only works when there are no subdirectories involved. Otherwise it tells me the file does not exist. Simply trying to get python to see the csv files in those subdirectories and run them through the program.

For example, this works just fine:

top = r'C:\Users\Brock\Desktop\Masters_Python&Data\WRF_data\ctl\2006\01'
os.chdir(top)    #change current directory to 'top' object
for root,dirs,files in os.walk(top, topdown = True):
    for file in files:
    #blah blah program goes here

However when I try to run this:

top = r'C:\Users\Brock\Desktop\Masters_Python&Data\WRF_data\ctl'
os.chdir(top)
for root,dirs,files in os.walk(top, topdown = True):    
    for file in files:
    #blah blah program goes here

It tells me that the first file in the first folder (01) does not exist. Also throws a random "b" in there after the word "File":

FileNotFoundError: File b'1136072700_KSUN_wrf6x6.csv' does not exist 

This occurs if top is something like "r'C:\\Users\\Brock\\Desktop\\Masters_Python&Data\\WRF_data\\ctl\\2006' as well.

Again, I apologize if this is a stupid question, but I'd rather ask someone who knows rather than me fiddling with it for a week to get somewhere.

Cheers

Answer 1

You need to use glob :

import glob
import csv

path = r'C:\Users\Brock\Desktop\Masters_Python&Data\WRF_data\ctl\*\*\*.csv'
for file in glob.iglob(path):
   with open(file) as f:
     reader = csv.reader(f, delimiter=',')
     for row in reader:
         do_something_with(row)