Combining CSV files into 1 CSV

Question

I am combining multiple CSV's in a folder into one CSV and for some reason when I run the code it keeps copying the same files into the 1 and repeating itself. Sorry if that is vague. Here's the code that I am running, this will probably explain it better than I did.

import csv
import glob

fo = open("CombinedLog.csv", "ab")

list = glob.glob('*.csv')
print list

for file in list:
  ifile  = open(file, "rb")
  reader = csv.reader(ifile)
  for row in reader:
    row = ",".join(row) + "\n"
    fo.write(row)
ifile.close()

fo.close()

Answer 1

You opened the CombinedLog.csv for appending ; every time you run this script more data will be added to the end, and the old data already there is retained.

Your code is working fine otherwise, but I'd use csv.writer() to format the output rows instead of using ','.join() ; you can feed the csv.reader() object directly into the csv.writer() .writerows() method and be done with it.

I'd also use the file objects as context managers with the with statement and have them closed automatically:

import csv
import glob

with open("CombinedLog.csv", "wb") as fo:
    fo_writer = csv.writer()

    csv_files = glob.glob('*.csv')
    for filename in csv_files:
    with open(file, "rb") as ifile:
        reader = csv.reader(ifile)
        fo_writer.writerows(reader)

Answer 2

If that is all you are doing, and you don't want a whole program for it... In windows command prompt from that directory, you can just run:

copy *.csv all.csv

This uses the wildcard * to copy all CSVs and creates a new file called "all.csv" that contains all of the data.

Edit: If you want inside of python:

import os
os.system('copy *.csv all.csv')