使用 Pandas，根据第二列的最小值从数据框中的一列（对于每组）获取值

Question

Let's assume we have a dataframe with 3 columns: the_customer , the_date , and the_amount . We need to create a dataframe that has, for each user, the_amount associated with the earliest / minimum value of the_date for each user. Here's what we're doing so far:

each_users_first_amount = our_data[['the_customer', 'the_date', 'the_amount']]\
    .sort_values(by='the_date', ascending = True)\
    .groupby('the_customer', as_index=False)\
    .apply(lambda x: x.head(1))\
    .rename(columns = { 'the_date': 'earliest_date', 'the_amount': 'first_amount' })

This approach technically works, however for some reason this function is operating very slow on our data, and I'm not sure which method in the chain is causing the function to run slow ( .apply ?). This also seems "hacky" in particular the line .apply(lambda x: x.head(1)) which uses head to grab the first row, which works because we previously sorted.

In particular, it would maybe be helpful if this could be done using .agg() in some way, since we are already using .agg() in another method chain to group the data and compute grouped-by metrics.

Answer 1

Using aggregate fuction is not efficient for dataframes of bigger size it consumes more time than interating while working on bigger dataframes. However in your code apply function or iterating is the only possible option so u cant replace it. But i think the time taking process in the code is sorting. Using sort after groupby might reduce the time complexity as sorting many small subsets of it will be easier than sorting the whole dataframe.