Concatenate list of dataframes

Question

This is my code snippet:

import os 
import pandas as pd

path = os.getcwd()
files = os.listdir(path)
df = []

for f in files:
    data = pd.read_csv(f, usecols = [0,1,2,3,4])
    df.append(data)

temp = pd.concat(df)

where df is a list of dataframes:

0
DataFrame
(1, 5)
1
DataFrame
(7, 5)
2
DataFrame
(5, 5)
3
DataFrame
(10, 5)
4
DataFrame
(1, 5)
5
DataFrame
(2, 5)

I'm trying to stack these dataframes below one another and get one single dataframe as output. I've tried a bunch of combinations from SO Q&A but none seems to work. I feel this is easy. What am I doing wrong?

Answer 1

You do not need a for loop or list comprehension for this task. Simply do:

pd.concat(df)

where df is the list of dataframes.

Here is an example:

import pandas as pd
import numpy as np

df1 = pd.DataFrame(np.random.randint(0,100,size=(1,5)), columns=list('ABCDE'))
df2 = pd.DataFrame(np.random.randint(0,100,size=(7,5)), columns=list('ABCDE'))
df3 = pd.DataFrame(np.random.randint(0,100,size=(5,5)), columns=list('ABCDE'))
df = [df1, df2, df3]

concatenated = pd.concat(df)

Yields (for example):

    A   B   C   D   E
0  10  48  49  84  86
0  29   5  44  20  80
1  80   7   5   9  81
2  35  32  15  42  33
3  59  79  74  80  66
4  48  91  44  33  73
5  52  98  94  44  86
6  70  16  73  25  71
0  52  20  75  34  90
1  92  88  26  35  26
2  54   3  49  70  46
3  24  12  71  69  57
4   3  71  93  58  74

And you can use .reset_index(drop=True) to reset the index if you desire.