[英]Summing rows in grouped pandas dataframe and return NaN
import pandas as pd
import numpy as np
d = {'l': ['left', 'right', 'left', 'right', 'left', 'right'],
'r': ['right', 'left', 'right', 'left', 'right', 'left'],
'v': [-1, 1, -1, 1, -1, np.nan]}
df = pd.DataFrame(d)
When a grouped dataframe contains a value of np.NaN
I want the grouped sum to be NaN
as is given by the skipna=False
flag for pd.Series.sum
and also pd.DataFrame.sum
however, this当分组数据帧包含np.NaN
值时,我希望分组总和为NaN
正如pd.Series.sum
和pd.DataFrame.sum
的skipna=False
标志给出的pd.Series.sum
,但是,这
In [235]: df.v.sum(skipna=False)
Out[235]: nan
However, this behavior is not reflected in the pandas.DataFrame.groupby
object但是,此行为并未反映在pandas.DataFrame.groupby
对象中
In [237]: df.groupby('l')['v'].sum()['right']
Out[237]: 2.0
and cannot be forced by applying the np.sum
method directly并且不能通过直接应用np.sum
方法来强制
In [238]: df.groupby('l')['v'].apply(np.sum)['right']
Out[238]: 2.0
I can workaround this by doing我可以通过这样做来解决这个问题
check_cols = ['v']
df['flag'] = df[check_cols].isnull().any(axis=1)
df.groupby('l')['v', 'flag'].apply(np.sum).apply(
lambda x: x if not x.flag else np.nan,
axis=1
)
but this is ugly.但这很丑陋。 Is there a better method?有没有更好的方法?
I think it's inherent to pandas.我认为这是熊猫固有的。 A workaround can be :解决方法可以是:
df.groupby('l')['v'].apply(array).apply(sum)
to mimic the numpy way,模仿麻木的方式,
or或者
df.groupby('l')['v'].apply(pd.Series.sum,skipna=False) # for series, or
df.groupby('l')['v'].apply(pd.DataFrame.sum,skipna=False) # for dataframes.
to call the good function.调用好函数。
I'm not sure where this falls on the ugliness scale, but it works:我不确定这属于丑陋程度,但它有效:
>>> series_sum = pd.core.series.Series.sum
>>> df.groupby('l')['v'].agg(series_sum, skipna=False)
l
left -3
right NaN
Name: v, dtype: float64
I just dug up the sum
method you used when you took df.v.sum
, which supports the skipna
option:我刚刚挖出你在取df.v.sum
时使用的sum
方法,它支持skipna
选项:
>>> help(df.v.sum)
Help on method sum in module pandas.core.generic:
sum(axis=None, skipna=None, level=None, numeric_only=None, **kwargs) method
of pandas.core.series.Series instance
Is that what you want?那是你要的吗?
In [24]: df.groupby('l')['v'].agg(lambda x: np.nan if x.isnull().any() else x.sum())
Out[24]:
l
left -3.0
right NaN
Name: v, dtype: float64
or或者
In [22]: df.groupby('l')['v'].agg(lambda x: x.sum() if x.notnull().all() else np.nan)
Out[22]:
l
left -3.0
right NaN
Name: v, dtype: float64
df.groupby(xxx).yyy.apply(lambda x: x.sum(skipna=False))
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.