How to concatenate multiple csv files into a single csv file using a column as index using python

Question

I have to merge different csv files which contain features about a place based on place_id into one so that I can create a model to predict a rating for a particular place.

I have already tried using pandas.concat and merging the files through linux terminal but I just get null values for all the other features as the place_id keeps on repeating

#importing libraries
import pandas as pd
import numpy as np
import glob


#creating a single dataframe
fileList = glob.glob('chef*.csv')
fileList.append('rating_final.csv')
dfList = []
for file in fileList:
    print(file)
    df = pd.read_csv(file)
    dfList.append(df)

concatDf = pd.concat(dfList, axis=0)

I expect to get a csv file with different features according to a single place_id but what I get is a csv file in which place_id keeps on repeating with a single feature only.

Answer 1

Try this,

import pandas as pd
df2 = pd.read_csv('rating_final.csv')
df2.to_csv('chef*.csv', mode='a', header=False, index=False)
test_df = pd.concat([pd.read_csv('chef*.csv'), df2], ignore_index=True, sort=True)
print(test_df)

The merged output will be available in chef*.csv file.