Reading several datasets

Question

I need to read 8 datasets:

df01, df02, df03, df04, df05, df06, df07, df08

This is my current approach:

#Set up file paths

filepath_2_df01= "folderX/df01.csv"
filepath_2_df02= "folderX//df02.csv"
filepath_2_df03= "folderX//df03.csv"
filepath_2_df04= "folderX//df04.csv"

filepath_2_df05= "folderY/df05.csv"
filepath_2_df06= "folderY/df06.csv"
filepath_2_df07= "folderY/df07.csv"
filepath_2_df08= "folderY/df08.csv"

#Read files

df01= pd.read_csv(filepath_2_df01)
df02= pd.read_csv(filepath_2_df02)
df03= pd.read_csv(filepath_2_df03)
df04= pd.read_csv(filepath_2_df04)

df05= pd.read_csv(filepath_2_df05)
df06= pd.read_csv(filepath_2_df06)
df07= pd.read_csv(filepath_2_df07)
df08= pd.read_csv(filepath_2_df08)

Is there a more concise way of doing that?

Answer 1

You can use glob for this:

import glob

dfs = []
for file in glob.glob('folderX/*.csv'):
    dfs.append(pd.read_csv(file))

for df in dfs:
    print(df)

Answer 2

use glob

from glob import glob

filenames = glob('/**/df*.csv') #get list of filenames, starting with df, in a specific folder. ** will search for subfolders as well(folderX and folderY, in your case)

dataframes = [pd.read_csv(f) for f in filenames] # using list comprehension, get a list of dataframes.
master_df = pd.concat(dataframes) #Concatenate list of dataframes to get a master dataframe