Pandas name dataframe from a string in csv name

Question

I have several csv with a string in their name (eg city name) and want to read them in dataframe with the names derived from that city name.

example of csv names: data_paris.csv, data_berlin.csv

How can I read them in a loop to get df_paris and df_berlin?

What I tried so far:

all_files = glob.glob(./*.csv")

for filename in all_files:
    city_name=re.split("[_.]", filename)[1] #to extract city name from filename
    dfname= {'df' + str(city_name)}
    print(dfname)
    dfname= pd.read_csv(filename)

I expect to have df_rome and df_paris, but I get just dfname. Why?

A related question: Name a dataframe based on csv file name?

Thank you!

Answer 1

I would recommend against automatic dynamic naming like df_paris , df_berlin . Instead, you should do:

all_files = glob.glob("./*.csv")

# dictionary of dataframes
dfs = dict()
for filename in all_files:
    city_name=re.split("[_.]", filename)[1] # to extract city name from filename

    dfs[city_name] =  pd.read_csv(filename) # assign to the dataframe dictionary

Answer 2

You are mixing your concepts. If you want to reference dynamically data frames that have been loaded use a dict

all_files = glob.glob("./*.csv")

dfname={}
                      
for filename in all_files:
    city_name=re.split("[_.]", filename)[1] #to extract city name from filename
    dfname['df' + str(city_name)] = pd.read_csv(filename)
print(list(dfname.keys())

Answer 3

the only dataframe you're creating is "dfname." You just keep overwriting that each time you loop through. I guess you could do this using globals(), though honestly I'd probably just create a list or a dict of dataframes (as it seems others have suggested while I was typing this), or else create a named column for 'city' in a master dataframe that I just keep appending to. But, keeping with what you're specifically asking, you could probably do it like so:

all_files = glob.glob("./*.csv")

for filename in all_files:
    globals()[filename[5:-4]]=  pd.read_csv(filename)