[英]Apply a function pairwise on a pandas series
我有一个熊猫系列,其元素构成了Frozensets:
data = {0: frozenset({'apple', 'banana'}),
1: frozenset({'apple', 'orange'}),
2: frozenset({'banana'}),
3: frozenset({'kumquat', 'orange'}),
4: frozenset({'orange'}),
5: frozenset({'orange', 'pear'}),
6: frozenset({'orange', 'pear'}),
7: frozenset({'apple', 'banana', 'pear'}),
8: frozenset({'banana', 'persimmon'}),
9: frozenset({'apple'}),
10: frozenset({'banana'}),
11: frozenset({'apple'})}
tokens = pd.Series(data); tokens
0 (apple, banana)
1 (orange, apple)
2 (banana)
3 (orange, kumquat)
4 (orange)
5 (orange, pear)
6 (orange, pear)
7 (apple, banana, pear)
8 (persimmon, banana)
9 (apple)
10 (banana)
11 (apple)
Name: Tokens, dtype: object
我想成对应用一个函数。 例如, tokens.diff
给我连续行之间的设置差:
0 NaN
1 (orange)
2 (banana)
3 (orange, kumquat)
4 ()
5 (pear)
6 ()
7 (apple, banana)
8 (persimmon)
9 (apple)
10 (banana)
11 (apple)
Name: Tokens, dtype: object
我想要相同的东西,但我希望在连续的行上设置集合并集,而不是设置差异。 因此,我理想地希望:
0 NaN
1 (orange, apple, banana)
2 (banana, orange, apply)
3 (orange, kumquat, banana)
4 (orange, kumquat)
...
如何使用Pandas实现这一目标? 我知道我可以使用zip
和list comp来做到这一点,但希望有更好的方法。
几种方法
选项1]清单理解
In [3631]: pd.Series([x[0].union(x[1])
for x in zip(tokens, tokens.shift(-1).fillna(''))],
index=tokens.index)
Out[3631]:
0 (orange, banana, apple)
1 (orange, apple, banana)
2 (orange, kumquat, banana)
3 (orange, kumquat)
4 (orange, pear)
5 (orange, pear)
6 (orange, pear, banana, apple)
7 (persimmon, pear, banana, apple)
8 (apple, persimmon, banana)
9 (apple, banana)
10 (banana, apple)
11 (apple)
dtype: object
选项2] map
In [3632]: pd.Series(map(lambda x: x[0].union(x[1]),
zip(tokens, tokens.shift(-1).fillna(''))),
index=tokens.index)
Out[3632]:
0 (orange, banana, apple)
1 (orange, apple, banana)
2 (orange, kumquat, banana)
3 (orange, kumquat)
4 (orange, pear)
5 (orange, pear)
6 (orange, pear, banana, apple)
7 (persimmon, pear, banana, apple)
8 (apple, persimmon, banana)
9 (apple, banana)
10 (banana, apple)
11 (apple)
dtype: object
选项3]使用concat
并apply
In [3633]: pd.concat([tokens, tokens.shift(-1).fillna('')],
axis=1).apply(lambda x: x[0].union(x[1]), axis=1)
Out[3633]:
0 (orange, banana, apple)
1 (orange, apple, banana)
2 (orange, kumquat, banana)
3 (orange, kumquat)
4 (orange, pear)
5 (orange, pear)
6 (orange, pear, banana, apple)
7 (persimmon, pear, banana, apple)
8 (apple, persimmon, banana)
9 (apple, banana)
10 (banana, apple)
11 (apple)
dtype: object
计时
In [3647]: tokens.shape
Out[3647]: (60000L,)
In [3648]: %timeit pd.Series([x[0].union(x[1]) for x in zip(tokens, tokens.shift(-1).fillna(''))], index=tokens.index)
10 loops, best of 3: 35 ms per loop
In [3649]: %timeit pd.Series(map(lambda x: x[0].union(x[1]), zip(tokens, tokens.shift(-1).fillna(''))), index=tokens.index)
10 loops, best of 3: 40.9 ms per loop
In [3650]: %timeit pd.concat([tokens, tokens.shift(-1).fillna('')], axis=1).apply(lambda x: x[0].union(x[1]), axis=1)
1 loop, best of 3: 2.2 s per loop
不相关并且为了diff
In [3653]: %timeit tokens.diff()
10 loops, best of 3: 10.8 ms per loop
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.