Combining Multiple CSV files with Python

Question

I am trying to combine multiple csv files into 1 csv file in a python script. I want to skip writing the first 5 lines of each csv file. Having some trouble and I am new to Python. I have tried several examples that I have found but it seems to have trouble with the working directory. Here is my latest attempt:

import pandas as pd
import csv
import glob
import os

path = '//server01/tmp/'
files_in_dir = [f for f in os.listdir(path) if f.endswith('csv')]
count = 0
for filenames in files_in_dir:
    df = pd.read_csv(filenames)
    if count < 6:
            count += 1
            continue
    df.to_csv('out.csv', mode='a')

Any help would be appreciated. Thanks!

Answer 1

Try this:

import pandas as pd
import csv
import glob
import os

path = '//server01/tmp/'
files_in_dir = [os.path.join(path,f) for f in os.listdir(path) if f.endswith('csv')]
for filenames in files_in_dir:
    df = pd.read_csv(filenames, skiprows = 5)
    df.to_csv('out.csv', mode='a')

skiprows : number of lines to skip

nrows : number of rows of file to read