通过多个文件夹串联多个文件

Question

Im trying to create a single file out of multiple text files I have across multiple folders. This is my code for concatenating. It works only if the program file is placed in each folder:

        import os

        file_list = [each for each in cur_folder if each.endswith(".txt")]
        print file_list

        align_file = open("all_the_files.txt","w")

        seq_list = []

        for each_file in file_list:
                f_o = open(file_path,"r")
                seq = (f_o.read().replace("\n",""))
                lnth = len(seq)
                wholeseq = ">"+each_file+" | "+str(lnth)+" nt\n"+seq+"\n"
                align_file.write(wholeseq)
                print "done" 

Now I tried to edit to make sure that it automatically runs through the entire Data folder and then enters the subdirectories and concatenates all the files without me having to paste the program file in each folder. This is the edit.

    import os

    dir_folder = os.listdir("C:\Users\GAMER\Desktop\Data")

    for each in dir_folder:
            cur_folder = os.listdir("C:\\Users\\GAMER\\Desktop\\Data\\"+each)
            file_list = []

            file_list = [each for each in cur_folder if each.endswith(".txt")]
            print file_list

            align_file = open("all_the_files.txt","w")

            seq_list = []

            for each_file in file_list:

                f_o = open(file_path,"r")
                seq = (f_o.read().replace("\n",""))
                lnth = len(seq)
                wholeseq = ">"+each_file+" | "+str(lnth)+" nt\n"+seq+"\n"
                align_file.write(wholeseq)
                print "done" , cur_folder

However when I run this , I get an error on the first file of the folder saying no such file exists. I can seem to understand why, specifically since it names the file which is not "hardcoded". Any help will be appreciated.

If the code looks ugly to you feel free to suggested better ways to do it.

Answer 1

Jamie is correct - os.walk is most likely the function you need.

An example based on your use case:

for root, dirs, files in os.walk(r"C:\Users\GAMER\Desktop\Data"):
    for f in files:
        if f.endswith('.txt'):
            print(f)

This will print the name of every single file within every folder within the root directory passed in os.walk, as long as the filename ends in .txt.

Python's documentation is here: https://docs.python.org/2/library/os.html#os.walk