[英]Removing rows from pandas dataframe based on several columns
From a pandas dataframe, I want to remove the "rois" where half or more of the rows have for any of the columns s, b1 or b2 a value of below 50.从熊猫数据框中,我想删除“rois”,其中一半或更多的行对于任何列 s、b1 或 b2 的值都低于 50。
Here an example dataframe:这是一个示例数据框:
roi s b1 b2
4 40 60 70
4 60 40 80
4 80 70 60
5 60 40 60
5 60 60 60
5 60 60 60
Only the three rows corresponding to roi 5 should be left over (roi 4 has 2 out of 3 rows where at least one of the values of s, b1, b2 is below 50).只应保留与 roi 5 对应的三行(roi 4 有 3 行中的 2 行,其中 s、b1、b2 的值中至少有一个低于 50)。
I have this implemented already, but wonder if there is a shorter (ie. faster and cleaner) way to do this:我已经实现了这个,但想知道是否有更短(即更快更干净)的方法来做到这一点:
for roi in data.roi.unique():
subdata = data[data['roi']==roi];
subdatas = subdata[subdata['s']>=50];
subdatab1 = subdatas[subdatas['b1']>=50];
subdatab2 = subdatab1[subdatab1['b2']>=50]
if((subdatab2.size/10)/(subdata.size/10) < 0.5):
data = data[data['roi']!=roi];
You can do transform
:你可以做
transform
:
s = (data.set_index('roi') # filter `roi` out of later comparison
.lt(50).any(1) # check > 50 on all columns
.groupby('roi') # groupby
.transform('mean') # compute the mean
.lt(0.5) # make sure mean > 0.5
.values
)
data[s]
Output:输出:
roi s b1 b2
3 5 60 40 60
4 5 60 60 60
5 5 60 60 60
You can use multiple filter conditions all at once to avoid creating intermediate data frames (space complexity efficiency), example:您可以同时使用多个过滤条件以避免创建中间数据帧(空间复杂度效率),例如:
for roi in data.roi.unique():
subdata2 = data[(data['roi']==roi) &
(data['s']>=50) &
(data['b2']>=50)]
if (subdata2.size/10)/(data[data['roi']==roi].size/10) < 0.5:
data = data[data['roi']!=roi]
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.