Pandas - slice data and calculate averages

Question

I have a parcel delivery data sheet looks like below structure:

route_id      parcel_id   loading_time           other_fields
  X1          001         14:20 25/07/2019       ...
  X2          025         14:23 25/07/2019       ...
...         ...                    ...

I would like to compute the average of all parcel's weight appeared in every 10 minutes (0-10, 11-20, 21-30) by each route_id. So the result sheet I want looks like:

route_id        time_window                                         average_weight(kg)
   X1           870 (i.e. 14:20 - 14:30，only show UpperBound)      550 
   X1           880                                                 1020
   ...          ...                                                 ...

How to do this easily in Pandas or in SQL server?

Answer 1

If I understand you correctly, you want to perform aggregations by route_id at 10-minute intervals. Also your loading_time is a string. Convert it to Timestamp first.

The example below uses some mock data since there was no sample input data:

loading_times = np.random.choice(pd.date_range('2019-07-25 9:00', '2019-07-25 9:20', freq='T'), 10)
df = pd.DataFrame({
    'route_id': np.random.randint(1, 4, len(loading_times)),
    'weight': np.random.randint(1, 5, len(loading_times)),
    'loading_time': loading_times
})

Sample data (sorted):

route_id  weight        loading_time
       1       2 2019-07-25 09:00:00
       1       1 2019-07-25 09:07:00
       1       4 2019-07-25 09:10:00
       1       1 2019-07-25 09:12:00
       1       2 2019-07-25 09:13:00
       1       2 2019-07-25 09:15:00
       1       3 2019-07-25 09:19:00
       2       4 2019-07-25 09:03:00
       3       4 2019-07-25 09:04:00
       3       3 2019-07-25 09:17:00

Then group it:

def summarize(x):
    return pd.Series({
        'count': len(x),
        'avg_weight': x['weight'].mean()
    })

by = ['route_id', pd.Grouper(key='loading_time', freq='10T')]
df.groupby(by).apply(summarize)

Result:

                              count  avg_weight
route_id loading_time                          
1        2019-07-25 09:00:00    2.0         1.5
         2019-07-25 09:10:00    5.0         2.4
2        2019-07-25 09:00:00    1.0         4.0
3        2019-07-25 09:00:00    1.0         4.0
         2019-07-25 09:10:00    1.0         3.0