Taking output calculated from data in multiple text files and writing them into different columns of a CSV file

Question

I am calculating values(numbers) from two numbers in differing columns of a text file. Then I am iterating over multiple text files to do the same calculation. I need to write the output to different columns of a CSV file where each column corresponds to the calculations obtained from an individual text file. I more or less know how to iterate over different files but I don't know how to tell Python to write to a different column. Any guidance is appreciated.

Answer 1

You can use the fact that zip provides lazy iteration to do this pretty efficiently. You can define a simple generator function that yeilds a calculation for every line of the file it is initialized with. You can also use contextlib.ExitStack to manage your open files in a single context manager:

from contextlib import ExitStack
from csv import writer

def calc(line):
    # Ingest a line, do some calculations on it.
    # This is the function you wrote.

input_files = ['file1.txt', 'file2.txt', ...]

def calculator(file):
    """
    A generator function that will lazily apply the calculation
    to each line of the file it is initialized with.
    """
    for line in file:
        yield calc(line)

with open('output.csv', 'w') as output, ExitStack() as input_stack:
    inputs = [calculator(input_stack.enter_context(open(file))) for file in input_files]
    output_csv = writer(output)
    output_csv.wite_row(input_files)  # Write heading based on input files
    for row in zip(*inputs):
        output_csv.write_row(row)

The output in the CSV will be in the same order as the file names in input_files .