![](/img/trans.png)
[英]How to check if all the elements in list in one pandas column are present in another pandas column
[英]How to check if all the elements in list are present in pandas column
我有一个数据框和一个列表:
df = pd.DataFrame({'id':[1,2,3,4,5,6,7,8],
'char':[['a','b'],['a','b','c'],['a','c'],['b','c'],[],['c','a','d'],['c','d'],['a']]})
names = ['a','c']
只有当a
和c
都存在于char
列中时,我才想获取行。(这里的顺序无关紧要)
预期输出:
char id
1 [a, b, c] 2
2 [a, c] 3
5 [c, a, d] 6
我的努力
true_indices = []
for idx, row in df.iterrows():
if all(name in row['char'] for name in names):
true_indices.append(idx)
ids = df[df.index.isin(true_indices)]
这给了我正确的输出,但对于大型数据集来说太慢了,所以我正在寻找更有效的解决方案。
使用pd.DataFrame.apply
:
df[df['char'].apply(lambda x: set(names).issubset(x))]
输出:
id char
1 2 [a, b, c]
2 3 [a, c]
5 6 [c, a, d]
您可以从名称列表构建一个集合以加快查找速度,并使用set.issubset
检查集合中的所有元素是否都包含在列列表中:
names = set(['a','c'])
df[df['char'].map(names.issubset)]
id char
1 2 [a, b, c]
2 3 [a, c]
5 6 [c, a, d]
将列表理解与issubset
使用:
mask = [set(names).issubset(x) for x in df['char']]
df = df[mask]
print (df)
id char
1 2 [a, b, c]
2 3 [a, c]
5 6 [c, a, d]
Series.map
的另一个解决方案:
df = df[df['char'].map(set(names).issubset)]
print (df)
id char
1 2 [a, b, c]
2 3 [a, c]
5 6 [c, a, d]
性能取决于行数和匹配值的数量:
df = pd.concat([df] * 10000, ignore_index=True)
In [270]: %timeit df[df['char'].apply(lambda x: set(names).issubset(x))]
45.9 ms ± 2.26 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [271]: %%timeit
...: names = set(['a','c'])
...: [names.issubset(set(row)) for _,row in df.char.iteritems()]
...:
46.7 ms ± 5.51 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [272]: %%timeit
...: df[[set(names).issubset(x) for x in df['char']]]
...:
45.6 ms ± 1.26 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [273]: %%timeit
...: df[df['char'].map(set(names).issubset)]
...:
18.3 ms ± 2.96 ms per loop (mean ± std. dev. of 7 runs, 100 loops each)
In [274]: %%timeit
...: n = set(names)
...: df[df['char'].map(n.issubset)]
...:
16.6 ms ± 278 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
In [279]: %%timeit
...: names = set(['a','c'])
...: m = [name.issubset(i) for i in df.char.values.tolist()]
...:
19.2 ms ± 317 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
试试这个。
df['char']=df['char'].apply(lambda x: x if ("a"in x and "c" in x) else np.nan)
print(df.dropna())
输出:
id char
1 2 [a, b, c]
2 3 [a, c]
5 6 [c, a, d]
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.