How to read multiple files and load them into dataframe

Question

I have a list of csv's in a folder called 11 in C:\\1. All the data has same number of columns.

A.csv

aa    zz    1     AA  
aab   qq    3     FF
ca    qq    5     QQ

B.csv

aa    GG    09    VV
aab   HH    03    WW
ca    CC    0     UU

How do I read each file in that folder into the dataframe so that it has a empty column between each new data read from csv.

so that it looks like:

A     B     C     D     E    F     G     H     I
aa    zz    1     AA         aa    GG    09    VV
aab   qq    3     FF         aab   HH    03    WW
ca    qq    5     QQ         ca    CC    0     UU

Ie:

dfs = {i: pd.read_csv('C:\\1\\{}.csv'.format(i)) for i in ['a', 'b']}
print (dfs['a'])

Or:

import os
import pandas as pd
filelist = os.listdir(targetdir) 
df_list = [pd.read_table(file) for file in filelist]
big_df = pd.concat(df_list)


    FileNotFoundError: File b'b.csv' does not exist

Answer 1

You can add a spacer Data Frame on each file read, like this:

import os
import pandas as pd

# with directory name 1, located at ~/1:
dir_name = "1"
path = "~"
dfs = []

# with files A.csv, B.csv in ~/1 (e.g. ~/1/A.csv):
for fname in os.listdir(f"{path}/{dir_name}"):

    df = pd.read_csv(f"{path}/{dir_name}/{fname}", header=None)
    spacer = pd.DataFrame([" "]*len(df))
    dfs.append(df)
    dfs.append(spacer)
master = pd.concat(dfs, axis=1)

master
     0   1  2     3  0    0   1  2   3  0
0   aa  zz  1    AA      aa  GG  9  VV   
1  aab  qq  3    FF     aab  HH  3  WW   
2   ca  qq  5    QQ      ca  CC  0  UU   

If you really want the alphabetical column names ( A , B , C ) you specified in your post, use:

import string
colnames = string.ascii_uppercase
master.columns = [x for x in colnames[:len(master.columns)]]

Note: Directory path uses Mac syntax, but it should be easy enough to adapt to Windows.