[英]pivot irregular dictionary of lists into pandas dataframe
(或列表清单......我刚刚编辑过)
是否存在用于转换此类结构的现有python / pandas方法
food2 = {}
food2["apple"] = ["fruit", "round"]
food2["bananna"] = ["fruit", "yellow", "long"]
food2["carrot"] = ["veg", "orange", "long"]
food2["raddish"] = ["veg", "red"]
进入像这样的数据透视表?
+---------+-------+-----+-------+------+--------+--------+-----+
| | fruit | veg | round | long | yellow | orange | red |
+---------+-------+-----+-------+------+--------+--------+-----+
| apple | 1 | | 1 | | | | |
+---------+-------+-----+-------+------+--------+--------+-----+
| bananna | 1 | | | 1 | 1 | | |
+---------+-------+-----+-------+------+--------+--------+-----+
| carrot | | 1 | | 1 | | 1 | |
+---------+-------+-----+-------+------+--------+--------+-----+
| raddish | | 1 | | | | | 1 |
+---------+-------+-----+-------+------+--------+--------+-----+
天真的,我可能只是循环通过字典。 我看到如何在每个内部列表上使用地图,但我不知道如何在字典上加入/堆叠它们。 一旦我加入了它们,我就可以使用pandas.pivot_table了
for key in food2:
attrlist = food2[key]
onefruit_pairs = map(lambda x: [key, x], attrlist)
one_fruit_frame = pd.DataFrame(onefruit_pairs, columns=['fruit', 'attr'])
print(one_fruit_frame)
fruit attr
0 bananna fruit
1 bananna yellow
2 bananna long
fruit attr
0 carrot veg
1 carrot orange
2 carrot long
fruit attr
0 apple fruit
1 apple round
fruit attr
0 raddish veg
1 raddish red
纯蟒蛇:
from itertools import chain
def count(d):
cols = set(chain(*d.values()))
yield ['name'] + list(cols)
for row, values in d.items():
yield [row] + [(col in values) for col in cols]
测试:
>>> food2 = {
"apple": ["fruit", "round"],
"bananna": ["fruit", "yellow", "long"],
"carrot": ["veg", "orange", "long"],
"raddish": ["veg", "red"]
}
>>> list(count(food2))
[['name', 'long', 'veg', 'fruit', 'yellow', 'orange', 'round', 'red'],
['bananna', True, False, True, True, False, False, False],
['carrot', True, True, False, False, True, False, False],
['apple', False, False, True, False, False, True, False],
['raddish', False, True, False, False, False, False, True]]
[更新]
性能测试:
>>> from itertools import product
>>> labels = list("".join(_) for _ in product(*(["ABCDEF"] * 7)))
>>> attrs = labels[:1000]
>>> import random
>>> sample = {}
>>> for k in labels:
... sample[k] = random.sample(attrs, 5)
>>> import time
>>> n = time.time(); list(count(sample)); print time.time() - n
62.0367980003
在我忙碌的机器上花了不到2分钟,因为279936行乘1000列(打开了很多镀铬标签)。 如果表现不可接受,请告诉我。
[更新]
从另一个答案测试性能:
>>> n = time.time(); \
... df = pd.DataFrame(dict([(k, pd.Series(v)) for k,v in sample.items()])); \
... print time.time() - n
72.0512290001
下一行( df = pd.melt(...)
)花了太长时间,所以我取消了测试。 拿这个结果用一粒盐,因为它在繁忙的机器上运行。
使用熊猫的答案。
# Test data
food2 = {}
food2["apple"] = ["fruit", "round"]
food2["bananna"] = ["fruit", "yellow", "long"]
food2["carrot"] = ["veg", "orange", "long"]
food2["raddish"] = ["veg", "red"]
df = DataFrame(dict([ (k,Series(v)) for k,v in food2.items() ]))
# pivoting to long format
df = pd.melt(df, var_name='item', value_name='categ')
# cross-tabulation
df = pd.crosstab(df['item'], df['categ'])
# sorting index, maybe not necessary
df.sort_index(inplace=True)
df
categ fruit long orange red round veg yellow
item
apple 1 0 0 0 1 0 0
bananna 1 1 0 0 0 0 1
carrot 0 1 1 0 0 1 0
raddish 0 0 0 1 0 1 0
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.