How to read csv files in Python with incremental names and create different objects?

Question

I have lots of files name like "XXXXX_1.csv", "XXXX_2.csv","XXXX_3.csv"...."XXXX_n.csv" I would like to read them and create df1, df2, df3...... How should I do this? In R, I could write like

fname <- filename[i]
assign(paste0("dry_shell",i),fread(paste0("/mnt/Wendy/Data/",fname)))
}

But how about Python? I would like to have different dataframes like df1,df2,df3 that assign to dataframe1, dataframe2, etc.

Answer 1

Assuming they are all named nicely you can iterate over a range of numbers to get all the files:

# this will open and do something to all files named csv_0, csv_1, and csv_2 in the
# directory /path/to/files
for i in range(3):
    with open(f"/path/to/files/csv_{i}") as file:
        # do something with the csv file ...

You could also provide a path to a directory and achieve the same goal by opening and processing all the files in the dictionary:

import os

for path in os.listdir():
    with open(path) as file:
        # do something with the csv file ...

See here for the documentation for the python (v3.8.5) standard csv package.

Answer 2

From your question, I assume you are using datatable (I noticed the fread function). The equivalent, dataframe wise, in python is Pandas. You can combine pathlib with Pandas to create a dictionary of dataframes:

from pathlib import Path
directory = Path(directory that contains  "XXXXX_1.csv", "XXXX_2.csv","XXXX_3.csv"...."XXXX_n.csv")
{f"df{n}": pd.read_csv(file) for n, file in enumerate(directory.iterdir(),1)}