Read several files, and stack them into a single multilevel data frame. Each file has the same column names

Question

I wanted to ask if anyone knows how to use the multilevel index to stack several data frames into a single one instead of a list of data frames like the one I am doing. Thanks

import glob
import pandas as pd

glist=glob.glob("./path/*.csv")

D=[]
for file in glist:
    X=pd.read_csv(file,names=['name1','name2','name3'],index_col = 0, header=0)
    D.append(X)

Answer 1

look here for a good start, you need to put the paths in a list, than use pandas concat() to put them together

import pandas as pd
import os
from os import path
dfs = [pd.read_csv(path.join('data',x)) for x in os.listdir("data") if path.isfile(path.join("data",x))]
df = pd.concat(dfs)

If you like to assign new columns to a data frame use assign two join multiple dataframes based on multiple levels of index look here

and in order to combine two dataframes into one with hierachial column index, do something like:

 pd.concat(dict(df1 = df1, df2 = df2),axis=1)

also Pandas has a built in function to merge two data frames, look here

Answer 2

This seems to do what I wanted. Thank you Wen.

       D=pd.DataFrame()
       for file in glist:
               X=pd.read_csv(file,names=['name1','name2','name3'],header=0,index_col=0)
               D=pd.concat([X,D],axis=0)