Reading multiple log files from a folder using python

Question

In the past I have read a log file using python with the following code, which has always worked fine:

with open(r"24T23.log") as f, open('logfile.csv', 'w') as f2:
    writer = csv.writer(f2)
    writer.writerow(['Index','Date', 'Time', 'Logic', '(Logic)','Type', 'Code','Connector', 'Message', 'Extra 1', 'Extra 2'])

    i = 0
    for line in f:
        writer.writerow([i] + line.rstrip().split('\t'))
        i += 1

For a particular use case that I am working on, I need to read multiple files contained in a folder. Can someone please suggest how to modify the above code (I tried that using blob but could not succeed)? Thanks in advance.

Answer 1

How's this?

import csv
import os

# From root of cwd
DIRECTORY = 'test'

# This gets the path of every file in DIRECTORY if it is a log file
# os.path.join(os.getcwd(),x) to include cwd in directory name
logs = [os.path.join(os.getcwd(), x) for x in os.listdir(
    DIRECTORY) if os.path.splitext(x)[1] == '.log']

for log in logs:
    # Note the 'a' append write, to append the rows to the files
    with open(log) as f, open('logfile.csv', 'a') as f2:
        writer = csv.writer(f2)
        writer.writerow(['Index', 'Date', 'Time', 'Logic', '(Logic)',
                        'Type', 'Code', 'Connector', 'Message', 'Extra 1', 'Extra 2'])

        i = 0
        for line in f:
            writer.writerow([i] + line.rstrip().split('\t'))
            i += 1

Docs on os.listdir()