How can I add columns from 2 different files to an output in CSV python

Question

my test.csv

1,1,2
2,1,3
3,1,4

my test2.csv

2,3
2,3
2,3

How can i make the output.csv:

1,1,2,2,3
2,1,3,2,3
3,1,4,2,3

so to combine two csv files to one?

Here's my code

import csv, os, sys
with open('test.csv', 'rb') as input, open('output.csv', 'wb') as output, open ('test2.csv', 'rb') as input2:
        reader = csv.reader(input, delimiter = ',')
        reader2 = csv.reader(input2, delimiter = ',')
        writer = csv.writer(output, delimiter = ',')

        all = []                                        
        header = next(reader)
        all.append(header)
        count = 0
        for row,row2 in reader and reader2:
                count += 1
                while count:
                        all.append(row+row2)
                        break
        writer.writerows(all)

Obviously this doesn't work, but does anybody understand what I'm going for?

Answer 1

Use zip() to iterate over both readers at once:

reader1 = csv.reader(input, delimiter = ',')
reader2 = csv.reader(input2, delimiter = ',')

for row1, row2 in zip(reader1, reader2):
    writer.writerow(row1 + row2)

Or a shorter version:

writer.writerows(map(list.__add__, row1, row2))

In case the files are huge then using map , zip is not going to be a good idea in Python 2, as they will load all the rows from both files, better go for their iterator versions present in itertools module: itertools.imap and itertools.izip :

for row,row2 in reader and reader2: is equivalent to iterating over just reader2 because and works like this:

>>> 1 and 2 
2
>>> 2 and 3
3
>>> 0 and 2  # returned the first falsy value, but as an iterator is not a falsy value
0            # so it will return `reader2` in your case.

Update:

To update test2.csv in-place you can use the fileinput module, but with this you won't be able to use the csv module.

>>> import fileinput
>>> with open('test.csv') as f:
    for line in fileinput.input('test2.csv', inplace=True):
        print next(f).rstrip() + ',' + line,
...         
>>> !cat test2.csv
1,1,2,2,3
2,1,3,2,3
3,1,4,2,3

Using csv module you'll have to read all the lines from test2.csv in memory first and then write the new data into it.

with open('test.csv') as f1, open('test2.csv', 'r+') as f2:
                                   #open in r+ mode
   reader1 = csv.reader(f1)
   rows_f2 = list(csv.reader(f2)) #read all the rows
   f2.truncate(0)                 #truncate the file
   writer = csv.writer(f2)
   writer.writerows(map(list.__add__, reader1, rows_f2))

Answer 2

Simply just concatenate line by line with comma...

with open('test.csv', 'rb') as input, open('output.csv', 'wb') as output, open ('test2.csv', 'rb') as input2:
    for row, row2 in zip(input, input2):
        output.write(row.rstrip('\n') + ',' + row2)