How to read a lot of txt file in specific folder using python

Question

Please help me, i have some file txt in folder. I want to read and summary all data become one file txt. How can I do it with python. for example :

folder name : data
file name in that folder : log1.txt
                           log2.txt
                           log3.txt
                           log4.txt
data in log1.txt : Size:         1,116,116,306 bytes
data in log2.txt : Size:         1,116,116,806 bytes
data in log3.txt : Size:         1,457,116,806 bytes
data in log4.txt : Size:         1,457,345,000 bytes

My expected output:

   a file txt the result.txt and the data is : 1,116,116,306 
                                               1,116,116,806 
                                               1,457,116,806 
                                               1,457,345,000  

Answer 1

Did you mean you want to read the contents of each file and write all of them in to a different file.

import os
#returns the names of the files in the directory data as a list
list_of_files = os.listdir("data")
lines=[]
for file in list_of_files:
    f = open(file, "r")
    #append each line in the file to a list
    lines.append(f.readlines())
    f.close()

#write the files to result.txt
result = open("result.txt", "w")
result.writelines(lines)
result.close()

If you are looking for size of file instead of the contents. change the two lines :

 f= open(file,"r")
lines.append(f.readlines())

to:

lines.append(os.stat(file).st_size)

Answer 2

File concat.py

#!/usr/bin/env python
import sys, os

def main():
    folder = sys.argv[1] # argument contains path
    with open('result.txt', 'w') as result: # result file will be in current working directory
        for path in os.walk(folder).next()[2]: # list all files in provided path
            with open(os.path.join(folder, path), 'r') as source:
                result.write(source.read()) # write to result eachi file

main()

Usage concat.py <your path>

Answer 3

1. You have to find all files that you are going to read:

    path = "data" files = os.listdir(path) 

2. You have to read all files and for each of them to collect the size and the content:

    all_sz = {i:os.path.getsize(path+'/'+i) for i in files} all_data = ''.join([open(path+'/'+i).read() for i in files]) 

3. You need a formatted print:

    msg = 'this is ...;' sp2 = ' '*4 sp = ' '*len(msg) + sp2 print msg + sp2, for i in all_sz: print sp, "{:,}".format(all_sz[i]) 

Answer 4

Import os . Then list the folder contents using os.listdir('data') and store it in an array. For each entry you can get the size by calling os.stat(entry).st_size . Each of these entries can now be written to a file.

Combined:

import os

outfile = open('result.txt', 'w')
path = 'data'
files = os.listdir(path)
for file in files:
    outfile.write(str(os.stat(path + "/" + file).st_size) + '\n')

outfile.close()

Answer 5

If one needs to merge sorted files so that the output file is sorted too, they can use the merge method from the heapq standard library module.

from heapq import merge
from os import listdir

files = [open(f) for f in listdir(path)]
with open(outfile, 'w') as out:
    for rec in merge(*files):
        out.write(rec)

Records are kept sorted in lexical order, if one needs something different merge accepts a key=... optional argument to specify a different ordering function.