在熊猫GroupBy对象上“应用”的替代方法

Question

So I have the following:

timeDiffFunc = lambda x: x['CP_EX_DT'] - x['CP_EX_DT'].shift(1)
exTimeDiff = assetGrp.apply(timeDiffFunc).fillna(0).reset_index(level=1)

But this uses so much memory that my system crashes (similar to the issue seen here: Memory leak in Pandas.groupby.apply()? )

My question is, how can I convert this to code that does not use the apply function? I tried variations of:

for i, (name,grp) in enumerate(assetGrp):
  grp = grp['CP_EX_DT'] - grp['CP_EX_DT'].shift(1)
exTimeDiff = assetGrp.fillna(0).reset_index(level=1)

but always received an error like: NotImplementedError: Index._join_level on non-unique index is not implemented when trying to merge the result back into the dataframe.

Any advice would be greatly appreciated.

Answer 1

根据上面的Uvar的评论，assetGrp.diff（）以较低的内存开销执行了相同的操作。