[英]Pandas dataframe boolean mask on multiple columns
I have a dataframe (df) containing several columns with an actual measure and corresponding number of columns (A,B,...) with an uncertainty (dA, dB, ...) for each of these columns: 我有一个数据帧(df),其中包含几个具有实际测量值的列和相应的列数(A,B,...),每个列的不确定性(dA,dB,...):
A B dA dB
0 -1 3 0.31 0.08
1 2 -4 0.263 0.357
2 5 5 0.382 0.397
3 -4 -0.5 0.33 0.115
I apply a function to find values in the measurement columns that are valid according to my definition 我应用一个函数来根据我的定义在测量列中查找有效的值
df[["A","B"]].apply(lambda x: x.abs()-5*df['d'+x.name] > 0)
This will return a boolean array: 这将返回一个布尔数组:
A B
0 False True
1 True True
2 True True
3 True False
I would like to use this array to select rows in dataframe for which the condition is true within a single column, eg A -> row 1-3 , and also find rows where the condition is true for all the input columns, eg row 1 and 2 . 我想使用此数组在单个列中选择条件为真的数据帧中的行,例如A - >第1-3行,并且还查找所有输入列的条件为真的行,例如第1行和2 。 Is there an efficient way to do this with pandas? 有没有一种有效的方法来做大熊猫?
You can use the results of your apply statement to boolean index select from the original dataframe: 您可以将apply语句的结果用于从原始数据帧中选择布尔索引:
results = df[["A","B"]].apply(lambda x: x.abs()-5*df['d'+x.name] > 0)
Which returns your boolean array above: 返回上面的布尔数组:
A B
0 False True
1 True True
2 True True
3 True False
Now, you can use this array to select rows from your original datafame as follows: 现在,您可以使用此数组从原始数据名称中选择行,如下所示:
Select where A is True: 选择A为True的位置:
df[results.A]
A B dA dB
1 2 -4.0 0.263 0.357
2 5 5.0 0.382 0.397
3 -4 -0.5 0.330 0.115
Select where either A or B are true: 选择A或B为真的位置:
df[results.any(axis=1)]
A B dA dB
0 -1 3.0 0.310 0.080
1 2 -4.0 0.263 0.357
2 5 5.0 0.382 0.397
3 -4 -0.5 0.330 0.115
Select where all the columns true: 选择所有列为true的位置:
df[results.all(axis=1)]
A B dA dB
1 2 -4.0 0.263 0.357
2 5 5.0 0.382 0.397
Using the underlying array data, a vectorized approach would be like so - 使用底层数组数据,矢量化方法就是这样 -
cols = ['A','B'] # list holding relevant column names
dcols = ['d'+i for i in cols]
out = np.abs(df[cols].values) - 5*df[dcols].values > 0
Sample run - 样品运行 -
In [279]: df
Out[279]:
A B dA dB
0 -1 3.0 0.310 0.080
1 2 -4.0 0.263 0.357
2 5 5.0 0.382 0.397
3 -4 -0.5 0.330 0.115
In [280]: cols = ['A','B'] # list holding relevant column names
...: dcols = ['d'+i for i in cols]
...: out = np.abs(df[cols].values) - 5*df[dcols].values > 0
...:
In [281]: out
Out[281]:
array([[False, True],
[ True, True],
[ True, True],
[ True, False]], dtype=bool)
To extract out the valid ones by setting the invalid ones as NaNs
, we could use np.where
- 要通过将无效的NaNs
设置为NaNs
来提取有效的,我们可以使用np.where
-
In [293]: df[cols] = np.where(out, df[cols], np.nan)
In [294]: df
Out[294]:
A B dA dB
0 NaN 3.0 0.310 0.080
1 2.0 -4.0 0.263 0.357
2 5.0 5.0 0.382 0.397
3 -4.0 NaN 0.330 0.115
Also, we could get the rows with all matches with all()
reduction along each row - 此外,我们可以获得所有匹配的行以及每行的all()
减少 -
In [283]: np.flatnonzero(out.all(axis=1))
Out[283]: array([1, 2])
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.