[英]Pandas split CSV into multiple CSV's (or DataFrames) by a column
我很遗憾有一个问题,一些帮助或提示将不胜感激。
问题:我有一个csv文件,其列可能有多个值,如:
Fruit;Color;The_evil_column
Apple;Red;something1
Apple;Green;something1
Orange;Orange;something1
Orange;Green;something2
Apple;Red;something2
Apple;Red;something3
我已将数据加载到数据帧中,我需要根据“The_evil_column”列的值将该数据帧拆分为多个数据帧:
df1
Fruit;Color;The_evil_column
Apple;Red;something1
Apple;Green;something1
Orange;Orange;something1
df2
Fruit;Color;The_evil_column
Orange;Green;something2
Apple;Red;something2
df3
Fruit;Color;The_evil_column
Apple;Red;something3
阅读一些帖子后我更加困惑,我需要一些关于此的提示。
您可以生成DataFrames的字典:
d = {g:x for g,x in df.groupby('The_evil_column')}
In [95]: d.keys()
Out[95]: dict_keys(['something1', 'something2', 'something3'])
In [96]: d['something1']
Out[96]:
Fruit Color The_evil_column
0 Apple Red something1
1 Apple Green something1
2 Orange Orange something1
或DataFrames列表:
In [103]: l = [x for _,x in df.groupby('The_evil_column')]
In [104]: l[0]
Out[104]:
Fruit Color The_evil_column
0 Apple Red something1
1 Apple Green something1
2 Orange Orange something1
In [105]: l[1]
Out[105]:
Fruit Color The_evil_column
3 Orange Green something2
4 Apple Red something2
In [106]: l[2]
Out[106]:
Fruit Color The_evil_column
5 Apple Red something3
更新:
In [111]: g = pd.read_csv(filename, sep=';').groupby('The_evil_column')
In [112]: g.ngroups # number of unique values in the `The_evil_column` column
Out[112]: 3
In [113]: g.apply(lambda x: x.to_csv(r'c:\temp\{}.csv'.format(x.name)))
Out[113]:
Empty DataFrame
Columns: []
Index: []
将产生3个文件:
In [115]: glob.glob(r'c:\temp\something*.csv')
Out[115]:
['c:\\temp\\something1.csv',
'c:\\temp\\something2.csv',
'c:\\temp\\something3.csv']
你可以按列的值过滤框架:
frame=pd.read_csv('file.csv',delimiter=';')
frame['The_evil_column']=='something1'
这会返回:
0 True
1 True
2 True
3 False
4 False
5 False
Name: The_evil_column, dtype: bool
因此,您可以访问这些列:
frame1 = frame[frame['The_evil_column']=='something1']
稍后您可以删除列:
frame1 = frame1.drop('The_evil_column', axis=1)
更简单但效率更低的方法是:
data = pd.read_csv('input.csv')
out = []
for evil_element in list(set(list(data['The_evil_column']))):
out.append(data[data['The_evil_column']==evil_element])
out
将包含所有数据数据帧的列表。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.