[英]Using a pandas dataframe as a lookup table
Given a single row from dataframe X
, what is the most efficient way to retrieve all rows from dataframe Y
that completely match the query row? 给定数据帧
X
单行,从数据框Y
中检索与查询行完全匹配的所有行的最有效方法是什么?
Example: querying row [0,1,0,1]
from 示例:
[0,1,0,1]
查询行[0,1,0,1]
[
[0,1,0,1, 1.0],
[0,1,0,1, 2.0],
[0,1,0,0, 3.0],
[1,1,0,0, 0.5],
]
should return 应该回来
[
[0,1,0,1, 1.0],
[0,1,0,1, 2.0],
]
X
and Y
are assumed to have the same schema, except that Y
has an additional target value column. 假设
X
和Y
具有相同的模式,除了Y
具有附加目标值列。 There may be one, zero, or many matches. 可能有一个,零个或多个匹配。 The solution should be efficient even with thousands of columns.
即使有数千列,该解决方案也应该是高效的。
Use boolean indexing
: 使用
boolean indexing
:
L = [
[0,1,0,1, 1.0],
[0,1,0,1, 2.0],
[0,1,0,0, 3.0],
[1,1,0,0, 0.5],
]
df = pd.DataFrame(L)
Y = [0,1,0,1]
print (df[df.iloc[:, :len(Y)].eq(Y).all(axis=1)])
0 1 2 3 4
0 0 1 0 1 1.0
1 0 1 0 1 2.0
Explanation : 说明 :
First select first N
columns by length of sequence: 首先按序列长度选择前
N
列:
print (df.iloc[:, :len(Y)])
0 1 2 3
0 0 1 0 1
1 0 1 0 1
2 0 1 0 0
3 1 1 0 0
Compare all rows by first row selected by eq
and loc
: 比较
eq
和loc
选择的第一行的所有行:
print (df.iloc[:, :len(Y)].eq(Y))
0 1 2 3
0 True True True True
1 True True True True
2 True True True False
3 False True True False
And check if match by DataFrame.all
for check all True
s per row: 并检查是否匹配
DataFrame.all
以检查每行的所有True
:
print (df.iloc[:, :len(Y)].eq(Y).all(1))
0 True
1 True
2 False
3 False
dtype: bool
One efficient way is to drop down to numpy
and query individual columns: 一种有效的方法是下拉到
numpy
并查询单个列:
Data from @jezrael. 来自@jezrael的数据。
import pandas as pd, numpy as np
df = pd.DataFrame({'A':list('abadef'),
'B':[4,5,4,5,5,4],
'C':[7,8,7,4,2,3],
'D':[1,3,1,7,1,0],
'E':[5,3,5,9,2,4],
'F':list('aaabbb')})
vals = df.values
arr = [4, 7, 1, 5]
mask = np.logical_and.reduce([vals[:, i+1]==arr[i] for i in range(len(arr))])
res = df.iloc[np.where(mask)[0]]
print(res)
# A B C D E F
# 0 a 4 7 1 5 a
# 2 a 4 7 1 5 a
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.