Looping over columns in Python and writing the output for each column in a file

Question

I have 7 columns in my dataset. A part of my script is taking the columns and processing it. For example the following is working on second column

for line in f:
    input_list.append(float(line.split()[1]))

I want it to process all 7 columns and writing each output as 'file$columnno.dat'

Question 1 : Is this a correct way to do it?

mylist = [1, 2, 3, 4 , 5, 6, 7]
for n in my list:
    for line in f:
        input_list.append(float(line.split()[n]))

Question 2 : Now the output is just a list of numbers.

print(*closed, sep='\n')

But I want the output for each column as a file such as file1.dat (1 is the same syntax of the column no.), file2.dat etc. Is that %f command. I didn't manage to fix it. It seems pretty standard and sorry if I am overwriting this question with existing ones.

Answer 1

Looks like you need list.extend

Ex:

for line in f:
    input_list.extend( map(float, line.split()) )

• Using map to convert every element in list to float

Answer 2

Question 1

Your solution will not work, because you can't iterate over the same line twice, unless you use seek(0) (see docs: Methods of File Objects ) to start again from the first line. Instead, you can iterate each line and create a list of lists, with each sublist representing a row in your file.

The csv module makes the syntax easier, so you don't need to manually iterate, split strings or convert to float . These can be handled by csv.reader in an efficient way:

from io import StringIO
import csv

mystr = StringIO("""34.12 42.13 4.1 65.13 -42.314 54.1 45.32
0.35 65.13 76.13 17.1 -45.1 65.1 78.1""")

# replace mystr with open('file.txt', 'r')
with mystr as fin:
    reader = csv.reader(fin, delimiter=' ', quoting=csv.QUOTE_NONNUMERIC)
    L = list(reader)

print(L)

[[34.12, 42.13, 4.1, 65.13, -42.314, 54.1, 45.32],
 [0.35, 65.13, 76.13, 17.1, -45.1, 65.1, 78.1]]

Question 2

You can iterate over each index of your list of lists via zip . Then, within your loop, iterate over values in your column. The output will be 7 files each with a column from the original input file. Here's an example:

for idx, column in enumerate(zip(*L), 1):
    with open(r'c:\temp\out_{0}.csv'.format(idx), 'w', newline='') as myfile:
        writer = csv.writer(myfile)
        for val in column:
            writer.writerow([val])