reading all the files in a directory with specific extension using glob python

Question

I have a directory with further sub directories each having files with a specific extension. I am able to get the names of all the files using glob function:

for name in glob.glob('*/*[a-b]*'):
      print(os.path.basename(name))

that prints the name of files I have in all the sub directories:

PF44_aa
PF52_aa
PF95_aa
PF38_aa
PF45_aa
PF63_aa
PF68_aa
PF39_aa

However, if I pass these file names as arguments to open the files and read the contents:

for name in glob.glob('*/*[a-b]*'):
    filename=os.path.basename(name)
    with open('%s' %filename) as fn:
        content = fn.readlines() 

I get the following error:

 File "<ipython-input-194-147f38fc2684>", line 1, in <module>
    with open('%s' %filename) as fn:

FileNotFoundError: [Errno 2] No such file or directory: 'PF44_aa'

I also tried giving the filename directly as input instead of %s :

for name in glob.glob('*/*[a-b]*'):
    filename=os.path.basename(name)
    with open(filename) as fn:
        content = fn.readlines() 

But still got the same error:

File "<ipython-input-193-fb125b5aa813>", line 1, in <module>
    with open(filename) as fn:

FileNotFoundError: [Errno 2] No such file or directory: 'PF44_aa'

What am I doing wrong and how can I fix this?

Answer 1

You have to use complete path of the file to open it, you can't use just filename unless if its on the same directory as your python file. So you have to do little change in your script to make it work.

for name in glob.glob('*/*[a-b]*'):
    with open(name) as fn:
        content = fn.readlines()

filename is replaced by name.
here, "name" is complete path to your file.

Answer 2

Alternative method...

Start by first importing:

import shutil
import os

Then assign the directory to a list:

file_list = []    
file_list = os.listdir('C:/filepath')

Now distinguish between files:

files = []
files = [x for x in file_list if "_aa" in x]

Now you can open and read the files in the files list using your method.

however do:

filepath + filename
with open(filepath + filename) as fn:
content = fn.readlines() 

Currently you're just trying to open the file with its name, you need to include the full file path...

outcome:

"C:/filepath/PF44_aa"