[英]Collapsing rows with NaN entries in pandas dataframe
我有一个带有数据行的pandas DataFrame ::
# objectID grade OS method
object_id_0001 AAA Mac organic
object_id_0001 AAA Mac NA
object_id_0001 AAA NA organic
object_id_0002 NA NA NA
object_id_0002 ABC Win NA
即,同一对象ID通常有多个条目,但有时/通常条目具有NA。
因此,我只是在寻找一种可以在ObjectID上组合的方法,并报告非NA条目,例如上面的折叠到::
object_id_0001 AAA Mac organic
object_id_0002 ABC Win NA
这有效并且已经有很长一段时间了。 但是,有些人声称这是一个可能修复的错误。 正如当前实现的那样,如果每列存在,则first
返回第一个非null元素。
df.groupby('objectID', as_index=False).first()
objectID grade OS method
0 object_id_0001 AAA Mac organic
1 object_id_0002 ABC Win NaN
pd.concat
pd.concat([
pd.DataFrame([d.lookup(d.notna().idxmax(), d.columns)], columns=d.columns)
for _, d in df.groupby('objectID')
], ignore_index=True)
objectID grade OS method
0 object_id_0001 AAA Mac organic
1 object_id_0002 ABC Win NaN
stack
df.set_index('objectID').stack().groupby(level=[0, 1]).head(1).unstack()
grade OS method
objectID
object_id_0001 AAA Mac organic
object_id_0002 ABC Win None
如果偶然那些是字符串( 'NA'
)
df.mask(df.astype(str).eq('NA')).groupby('objectID', as_index=False).first()
一种替代方案,更具机械性
def aggregate(s):
u = s[s.notnull()].unique()
if not u.size: return np.nan
return u
df.groupby('objectID').agg(aggregate)
grade OS method
objectID
object_id_0001 AAA Mac organic
object_id_0002 ABC Win NaN
这将工作bfill
+ drop_duplicates
df.groupby('objectID',as_index=False).bfill().drop_duplicates('objectID')
Out[939]:
objectID grade OS method
0 object_id_0001 AAA Mac organic
3 object_id_0002 ABC Win NaN
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.