[英]concat + groupby + apply in multiple columns in panda dataframe
我有以下两个数据框:
df1
ticker date return high_low turnover
CRM 2017-01-03 0.018040 0.026957 5.722346e+08
MSFT 2017-01-03 -0.003344 0.011428 1.295037e+09
CRM 2017-01-04 0.024198 0.032646 6.762756e+08
MSFT 2017-01-04 -0.002881 0.010142 1.329482e+09
CRM 2017-01-05 -0.000275 0.015580 3.417927e+08
df2:
ticker date return high_low turnover
CRM 2017-01-03 0.018040 0.026957 5.722318e+08
MSFT 2017-01-03 -0.003344 0.011509 1.295037e+09
CRM 2017-01-04 0.024198 0.032575 6.761264e+08
MSFT 2017-01-04 -0.002881 0.010142 1.329480e+09
CRM 2017-01-05 -0.000275 0.015580 3.417930e+08
我有以下代码工作。 但我想可以将最后四行简化为一行。 在一行中执行三列的 concat+groupby+apply。
def get_min_absvalue(values):
return min(values, key = abs)
#simplify the following 4 lines in 1?
consolidated_return=(pd.concat((df1,df2),ignore_index=True,sort=False).groupby(['date','ticker'])['return'].apply(lambda x: get_min_absvalue(x)).reset_index())
consolidated_high_low=(pd.concat((df1,df2),ignore_index=True,sort=False).groupby(['date','ticker'])['high_low'].apply(lambda x: get_min_absvalue(x)).reset_index())
consolidated_turnover=(pd.concat((df1,df2),ignore_index=True,sort=False).groupby(['date','ticker'])['turnover'].apply(lambda x: get_min_absvalue(x)).reset_index())
merged = consolidated_return.merge(consolidated_high_low, on=['date', 'ticker']).merge(consolidated_turnover, on=['date', 'ticker'])
那可能吗?
df1
和df2
列:list
used_cols = ['return', 'high_low', 'turnover']
df_list = [pd.concat((df1, df2), ignore_index=True, sort=False).groupby(['date', 'ticker'])[v].apply(lambda x: min(x, key=abs)) for v in used_cols]
df_list
内容:pandas.core.series.Series
类型df_list[0]
date ticker
2017-01-03 CRM 0.026957
MSFT 0.011428
2017-01-04 CRM 0.032575
MSFT 0.010142
2017-01-05 CRM 0.015580
Name: high_low, dtype: float64
concat
而不是merge
:df_list
与concat
结合使用merged = pd.concat(df_list, axis=1).reset_index()
merged_new == merged_old
used_cols = ['return', 'high_low', 'turnover']
df_list = [pd.concat((df1, df2), ignore_index=True, sort=False).groupby(['date', 'ticker'])[v].apply(lambda x: min(x, key=abs)) for v in used_cols]
merged = pd.concat(df_list, axis=1).reset_index()
考虑一个简单的min
聚合调用:
agg_df = (pd.concat([df1,df2], ignore_index=True, sort=False)
.groupby(['date','ticker'], as_index=False)
.min()
)
print(agg_df)
# date ticker return high_low turnover
# 0 2017-01-03 CRM 0.018040 0.026957 5.722318e+08
# 1 2017-01-03 MSFT -0.003344 0.011428 1.295037e+09
# 2 2017-01-04 CRM 0.024198 0.032575 6.761264e+08
# 3 2017-01-04 MSFT -0.002881 0.010142 1.329480e+09
# 4 2017-01-05 CRM -0.000275 0.015580 3.417927e+08
print(merged.eq(agg_df))
# date ticker return high_low turnover
# 0 True True True True True
# 1 True True True True True
# 2 True True True True True
# 3 True True True True True
# 4 True True True True True
对于真正的绝对值最小聚合:
agg_df = (pd.concat([df1,df2], ignore_index=True, sort=False)
.groupby(['date','ticker'], as_index=False)['return', 'high_low', 'turnover']
.apply(lambda x: x.abs().min())
.reset_index()
)
print(agg_df)
# date ticker return high_low turnover
# 0 2017-01-03 CRM 0.018040 0.026957 5.722318e+08
# 1 2017-01-03 MSFT 0.003344 0.011428 1.295037e+09
# 2 2017-01-04 CRM 0.024198 0.032575 6.761264e+08
# 3 2017-01-04 MSFT 0.002881 0.010142 1.329480e+09
# 4 2017-01-05 CRM 0.000275 0.015580 3.417927e+08
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.