Appending to a .csv File in Python

Question

I am trying to use Python to write to an existing .csv file but instead of appending the data to the bottom of the existing file, I would like to append the new information to a new column of the .csv file. The reason for this is that I am wanting to "infinitely" loop through a read data section of code and then add the data read during each loop iteration to a new column, instead of row. In essence, I would like something as follows:

Row1Iteration1, Row1Iteration2, Row1Iteration3,..., Row1IterationX
Row2Iteration1, Row2Iteration2, Row2Iteration3,..., Row2IterationX
Row3Iteration1, Row3Iteration2, Row3Iteration3,..., Row3IterationX
Row4Iteration1, Row4Iteration2, Row4Iteration3,..., Row4IterationX
Row5Iteration1, Row5Iteration2, Row5Iteration3,..., Row5IterationX

Instead of:

Row1Iteration1
Row2Iteration1
Row3Iteration1
Row4Iteration1
Row5Iteration1
Row1Iteration2
Row2Iteration2
Row3Iteration2
etc...

Does anyone know of a way to do this or is this an impossible constraint of .csv files? Is there another way, besides .csv, that I may be able to do this and still manipulate with Excel?

Answer 1

This is simply not possible on any system or file type that I know of. You'll have to read the data in, manipulate it and save/overwrite the original file. This is just how file I/O works on pretty much any operating system. You can read files, (over)write files and append to the end of files, that's all.

If your file is very big, you could read it in piece by piece, make your manipulations and write those to a temporary file. Once you're done, the original file can be replaced with the new temporary file.

Answer 2

I hope this example helps you:

# of course you should read the file someway
s = """
Row1Iteration1, Row1Iteration2, Row1Iteration3
Row2Iteration1, Row2Iteration2, Row2Iteration3
Row3Iteration1, Row3Iteration2, Row3Iteration3
Row4Iteration1, Row4Iteration2, Row4Iteration3
Row5Iteration1, Row5Iteration2, Row5Iteration3
"""
ncols = 3

iterations = []
for line in s.strip().split('\n'):
    iterations.append([l.strip() for l in line.split(',')])

with open('output.csv', 'w') as out:
    for col in xrange(ncols):
        for it in iterations:
            print it[col]
            out.write(col)

Answer 3

As Chinmay said your best bet might be to read in the csv file, massage it around, then output it back to csv. Take a look at this stackoverflow discussion that seems to address something similar to what you are asking about.