[英]Calculate nunique() for groupby in pandas
我有一個包含列的數據框:
diff
- 注冊日期和付款日期之間的diff
,以天為單位 country
- 用戶的國家 user_id
campaign_id
- 另一個分類列,我們將在groupby中使用它 我需要計算每個country
不同用戶的數量+具有diff
<= n的campaign_id
群組。 例如,對於country
'A', campaign
'abc'和diff
7我需要計算來自country
''', campaign
'abc'和diff
<= 7的不同用戶
我目前的解決方案(下面)工作時間太長
import pandas as pd
import numpy as np
## generate test dataframe
df = pd.DataFrame({
'country':np.random.choice(['A', 'B', 'C', 'D'], 10000),
'campaign': np.random.choice(['camp1', 'camp2', 'camp3', 'camp4', 'camp5', 'camp6'], 10000),
'diff':np.random.choice(range(10), 10000),
'user_id': np.random.choice(range(1000), 10000)
})
## main
result_df = pd.DataFrame()
for diff in df['diff'].unique():
tmp_df = df.loc[df['diff']<=diff,:]
tmp_df = tmp_df.groupby(['country', 'campaign'], as_index=False).apply(lambda x: x.user_id.nunique()).reset_index()
tmp_df['diff'] = diff
tmp_df.columns=['country', 'campaign', 'unique_ppl', 'diff']
result_df = pd.concat([result_df, tmp_df],ignore_index=True, axis=0)
也許有更好的方法來做到這一點?
首先使用列表理解與concat
並assign
加入所有在一起,然后groupby
與nunique
添加列diff
,最后重命名列,如果需要,為自定義列順序添加reindex
:
df1 = pd.concat([df.loc[df['diff']<=x].assign(diff=x) for x in df['diff'].unique()])
df2 = (df1.groupby(['diff','country', 'campaign'], sort=False)['user_id']
.nunique()
.reset_index()
.rename(columns={'user_id':'unique_ppl'})
.reindex(columns=['country', 'campaign', 'unique_ppl', 'diff']))
下面是一個替代方案,但@ jezrael的解決方案是最佳選擇。
績效基准
%timeit original(df) # 149ms
%timeit jp(df) # 81ms
%timeit jez(df) # 47ms
def original(df):
result_df = pd.DataFrame()
for diff in df['diff'].unique():
tmp_df = df.loc[df['diff']<=diff,:]
tmp_df = tmp_df.groupby(['country', 'campaign'], as_index=False).apply(lambda x: x.user_id.nunique()).reset_index()
tmp_df['diff'] = diff
tmp_df.columns=['country', 'campaign', 'unique_ppl', 'diff']
result_df = pd.concat([result_df, tmp_df],ignore_index=True, axis=0)
return result_df
def jp(df):
result_df = pd.DataFrame()
lst = []
lst_append = lst.append
for diff in df['diff'].unique():
tmp_df = df.loc[df['diff']<=diff,:]
tmp_df = tmp_df.groupby(['country', 'campaign'], as_index=False).agg({'user_id': 'nunique'})
tmp_df['diff'] = diff
tmp_df.columns=['country', 'campaign', 'unique_ppl', 'diff']
lst_append(tmp_df)
result_df = result_df.append(pd.concat(lst, ignore_index=True, axis=0), ignore_index=True)
return result_df
def jez(df):
df1 = pd.concat([df.loc[df['diff']<=x].assign(diff=x) for x in df['diff'].unique()])
df2 = (df1.groupby(['diff','country', 'campaign'], sort=False)['user_id']
.nunique()
.reset_index()
.rename(columns={'user_id':'unique_ppl'})
.reindex(columns=['country', 'campaign', 'unique_ppl', 'diff']))
return df2
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.