Merging a specific column from multiple text files into one file

Question

I have multiple folders and a text file ( input.txt ) in each folder. First I read the folder names from f_names.txt , then enter into each folder and read the third column from the input.txt in each folder. The code works properly until here. The problem is that the code merges all the third columns in a row in output file ( combine.txt ). Whereas I want to write each third column to the output file ( combine.txt ) as a new column. How can I do this?

Here is my code:

#!/usr/bin/python
import os
import re

path=os.getcwd()

try:
    os.remove("combine.txt")
except OSError:
    pass

with open('combine.txt', mode='a') as outfile:
    with open('f_names.txt', 'r') as read_f:
        for line in read_f:
            os.chdir(line.strip())
            with open('input.txt', 'r') as f:
                data=[]
                for line in f:
                    row = line.split()
                    data.append(float(row[2]))
                    outfile.write("%.2f\n" % float(row[2]))
            os.chdir("..")

Obtained output (for two input files):

2.12
3.15
4.18
8.45
2.10
0.12
0.18
0.32
0.21
0.13

Desired output (for two input files):

2.12 0.12
3.15 0.18
4.18 0.32
8.45 0.21
2.10 0.13

Answer 1

There are some things that you can do to make your program correct and "more pythonic."

with open('f_names.txt') as read_f:
    # Collect the third columns in a list
    data = []
    for line in read_f:
        # No need to chdir()
        with open('{}/input.txt'.format(line.strip())) as f:
            # This is one column
            data.append([float(line.split()[2]) for line in f])

# Do not remove the old file, overwrite it
with open('combine.txt', 'w') as outfile:    
    # "Transpose" the list of columns into a list of rows
    # Write each row
    for row in zip(*data):
        # Use new-style formatting
        outfile.write(" ".join("{:.2f}".format(cell) for cell in row) + "\n")