如何计算pandas数据帧上的连续有序值

Question

我试图从pandas上的数据框中获取来自给定数据帧的连续0值的最大计数，其中id，date，value列看起来像这样：

id    date       value
354   2019-03-01 0
354   2019-03-02 0
354   2019-03-03 0
354   2019-03-04 5
354   2019-03-05 5 
354   2019-03-09 7
354   2019-03-10 0
357   2019-03-01 5
357   2019-03-02 5
357   2019-03-03 8
357   2019-03-04 0
357   2019-03-05 0
357   2019-03-06 7
357   2019-03-07 7
540   2019-03-02 7
540   2019-03-03 8
540   2019-03-04 9
540   2019-03-05 8
540   2019-03-06 7
540   2019-03-07 5
540   2019-03-08 2 
540   2019-03-09 3
540   2019-03-10 2

所需的结果将按Id分组，如下所示：

id   max_consecutive_zeros
354  3
357  2
540  0

我已经实现了我想要的功能但是当你使用庞大的熊猫数据帧时它变得非常慢，我发现了一些类似的解决方案，但它根本不能解决我的问题。

Answer 1

这是我们需要创建额外的关键方法之一groupby然后，只需要groupby该键id

s=df.groupby('id').value.apply(lambda x : x.ne(0).cumsum())
df[df.value==0].groupby([df.id,s]).size().max(level=0).reindex(df.id.unique(),fill_value=0)
Out[267]: 
id
354    3
357    2
540    0
dtype: int64

Answer 2

你可以这样做：

df.groupby('id').value.apply(lambda x : ((x.diff() !=0).cumsum()).where(x ==0,\
                                       np.nan).value_counts().max()).fillna(0)

产量

id
354    3.0
357    2.0
540    0.0
Name: value, dtype: float64

Answer 3

为具有相同值的连续行创建groupID m 。 接下来， groupby的id和m和呼叫value_counts和.loc对多指标切片仅0最右边的指数水平的值。 最后，通过在id和reindex中duplicated来过滤掉重复索引，以便为没有0计数的id创建0值

m = df.value.diff().ne(0).cumsum().rename('gid')    
#Consecutive rows having the same value will be assigned same IDNumber by this command. 
#It is the way to identify a group of consecutive rows having the same value, so I called it groupID.

df1 = df.groupby(['id', m]).value.value_counts().loc[:,:,0].droplevel(-1)
#this groupby groups consecutive rows of same value per ID into separate groups.
#within each group, count number of each value and `.loc` to pick specifically only `0` because we only concern on the count of value `0`.

df1[~df1.index.duplicated()].reindex(df.id.unique(), fill_value=0)
#There're several groups of value `0` per `id`. We want only group of highest count. 
#`value_count` already sorted number of count descending, so we just need to pick 
#the top one of duplicates by slicing on True/False mask of `duplicated`.
#finally, `reindex` adding any `id` doesn't have value 0 in original `df`.
#Note: `id` is the column `id` in `df`. It is different from groupID `m` we create to use with groupby

Out[315]:
id
354    3
357    2
540    0
Name: value, dtype: int64