Going through all folders in Python

Question

I want to go through all folders inside a directory:

directory\  
   folderA\
         a.cpp
   folderB\
         b.cpp
   folderC\
         c.cpp
   folderD\
         d.cpp

The name of the folders are all known. Specifically, I am trying to count the number of lines of code on each of the a.cpp , b.cpp , c.pp and d.cpp source files. So, go inside folderA and read a.cpp , count lines and then go back to directory, go inside folderB , read b.cpp , count lines etc.

This is what I have up until now,

dir = directory_path
for folder_name in folder_list():
    dir = os.path.join(dir, folder_name)
    with open(dir) as file:
        source= file.read()
    c = source.count_lines()

but I am new to Python and have no idea if my approach is appropriate and how to proceed. Any example code shown will be appreciated!

Also, does the with open handles the file opening/closing as it should for all those reads or more handling is required?

Answer 1

I would do it like this:

import glob
import os

path = 'C:/Users/me/Desktop/'  # give the path where all the folders are located
list_of_folders = ['test1', 'test2']  # give the program a list with all the folders you need
names = {}  # initialize a dict

for each_folder in list_of_folders:  # go through each file from a folder
    full_path = os.path.join(path, each_folder)  # join the path
    os.chdir(full_path)  # change directory to the desired path

    for each_file in glob.glob('*.cpp'):  # self-explanatory
        with open(each_file) as f:  # opens a file - no need to close it
            names[each_file] = sum(1 for line in f if line.strip())

    print(names)

Output:

{'file1.cpp': 2, 'file3.cpp': 2, 'file2.cpp': 2}
{'file1.cpp': 2, 'file3.cpp': 2, 'file2.cpp': 2}

Regarding the with question, you don't need to close the file or make any other checks. You should be safe as it is now.

You may , however, check if the full_path exists as somebody (you) could mistakenly delete a folder from your PC (a folder from list_of_folders )

You can do this by os.path.isdir which returns True if the file exists:

os.path.isdir(full_path)

PS: I used Python 3.

Answer 2

Use Python 3's os.walk() to traverse all subdirectories and files of a given path, opening each file and do your logic. You can use a 'for' loop to walk it, simplifying your code greatly.

https://docs.python.org/2/library/os.html#os.walk

Answer 3

As manglano said, os.walk()

you can generate a list of folder.

[src for src,_,_ in os.walk(sourcedir)]

you can generate a list of file path.

[src+'/'+file for src,dir,files in os.walk(sourcedir) for file in files]