[英]How to create a dataframe from multiple csv files?
I am loading a csv file in pandas as我正在 Pandas 中加载一个 csv 文件作为
premier10 = pd.read_csv('./premier_league/pl_09_10.csv')
However, I have 20+ csv files, which I was hoping to load as separate dfs (one df per csv) using a loop and predefined names, something similar to:但是,我有 20 多个 csv 文件,我希望使用循环和预定义名称作为单独的 dfs(每个 csv 一个 df)加载,类似于:
import pandas as pd
file_names = ['pl_09_10.csv','pl_10_11.csv']
names = ['premier10','premier11']
for i in range (0,len(file_names)):
names[i] = pd.read_csv('./premier_league/{}'.format(file_names[i]))
(Note, here I provide only two csv files as example) Unfortunately, this doesn't work (no error messages, but the the pd dfs don't exist). (注意,这里我只提供了两个 csv 文件作为示例)不幸的是,这不起作用(没有错误消息,但 pd dfs 不存在)。
Any tips/links to previous questions would be greatly appreciated as I haven't found anything similar on Stackoverflow.任何提示/以前问题的链接将不胜感激,因为我在 Stackoverflow 上没有发现任何类似的东西。
pathlib
to set a Path, p
, to the files使用pathlib
设置文件的路径p
.glob
method to find the files matching the pattern使用.glob
方法查找与模式匹配的文件pandas.read_csv
使用pandas.read_csv
创建数据pandas.read_csv
pandas.concat
to create a single dataframe from all the files.或者,使用带有pandas.concat
的列表pandas.concat
从所有文件创建单个数据帧。for-loop
in the OP, objects (variables) may not be created in that way (eg names[i]
).在 OP 的for-loop
中,可能不会以这种方式创建对象(变量)(例如names[i]
)。
'premier10' = pd.read_csv(...)
, where 'premier10'
is a str
type.这相当于'premier10' = pd.read_csv(...)
,其中'premier10'
是str
类型。from pathlib import Path
import pandas as pd
# set the path to the files
p = Path('some_path/premier_league')
# create a list of the files matching the pattern
files = list(p.glob(f'pl_*.csv'))
# creates a dict of dataframes, where each file has a separate dataframe
df_dict = {f.stem: pd.read_csv(f) for f in files}
# alternative, creates 1 dataframe from all files
df = pd.concat([pd.read_csv(f) for f in files])
names = ['premier10','premier11']
does not create a dictionary but a list. names = ['premier10','premier11']
不会创建字典而是创建列表。 Simply replace it with names = dict()
or replace names = ['premier10','premier11']
by names.append(['premier10','premier11'])
只需将其替换为names = dict()
或将names = ['premier10','premier11']
names.append(['premier10','premier11'])
This is what you want:这就是你想要的:
#create a variable and look through contents of the directory
files=[f for f in os.listdir("./your_directory") if f.endswith('.csv')]
#Initalize an empty data frame
all_data = pd.DataFrame()
#iterate through files and their contents, then concatenate their data into the data frame initialized above
for file in files:
df = pd.read_csv('./your_directory' + file)
all_data = pd.concat([all_data, df])
#Call the new data frame and verify that contents were transferred
all_data.head()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.