Pandas generating an empty csv while trying combine all csv's into one csv

Question

I am writing a python script that will read all the csv files in the current location and merge them into a single csv file. Below is my code:-

import os
import numpy as np
import pandas as pd
import glob

path = os.getcwd()
extension = csv
os.chdir(path)
tables = glob.glob('*.{}'.format(extension))
data = pd.DataFrame()
for i in tables:
    try:
        df = pd.read_csv(r''+path+'/'+i+'')
        # Here I want to create an index column with the name of the file and leave that column empty
        df[i] = np.NaN
        df.set_index(i, inplace=True)
        # Below line appends an empty row for easy differentiation
        df.loc[df.iloc[-1].name+1,:] = np.NaN
        data = data.append(df)
    except Exception as e:
        print(e)
 data.to_csv('final_output.csv', indexx=False, header=None)

If I remove the below lines of code then it works:-

df[i] = np.NaN
df.set_index(i, inplace=True)

But I want to have the first column name as the name of the file and its values NaN or empty.

I want the output to look something like this:-

Answer 1

I tend to avoid the .append method in favor of pandas.concat

Try this:

import os
from pathlib import Path

import pandas as pd

files = Path(os.getcwd()).glob('*.csv')

df = pd.concat([
    pd.read_csv(f).assign(filename=f.name)
    for f in files
], ignore_index=True)

df.to_csv('alldata.csv', index=False)