pandas：获取Dataframe的每日描述

Question

I have a dataframe that looks like this:

        provider    timestamp                   vehicle_id
id          
103107  a           2019-09-11 20:05:47+02:00   x
1192195 b           2019-09-11 00:02:46+02:00   y
434508  c           2019-09-11 00:32:39+02:00   z
530388  c           2019-09-11 08:12:56+02:00   z
1773721 b           2019-09-11 20:02:55+02:00   w
...

I would like to get some statistics on the different vehicle_ids per day. I have this which allows me to do a describe manually:

df.groupby(['provider', df['timestamp'].dt.strftime('%Y-%m-%d')])[['vehicle_id']].nunique() :

                        vehicle_id
provider    timestamp   
a           2019-09-11  1224
            2019-09-12  1054
b           2019-09-11  2859
            2019-09-12  2761
            2019-09-17  700

How do I wrangle the data so I can get a daily min / max / average for each day? I'm kind of lost, any help is very appreciated.

Answer 1

Try this:

aggregations = ['mean', 'min', 'max', 'std']
result = grouped_df.groupby('timestamp')[vehicle_id].agg(aggregations)

Note: You might need to flatten your columns indexes first:

grouped_df.columns = [col[1] if col[1] != '' else col[0] for col in grouped_df.columns]

Answer 2

Try groupby().agg() :

new_df.groupby('timestamp').vehicle_id.agg({'min','max','mean'})

Note : Since you only care about one column in your original data, you can just pass a series in the first groupby instead of a data frame, ie,

# note the number of [] around 'vehicle_id'
new_df = (df.groupby(['provider', 
                     df['timestamp'].dt.strftime('%Y-%m-%d')])
          ['vehicle_id'].nunique()
         )

Then new_df is a series named vehicle_id , and the next command is just

# note the difference before .agg
new_df.groupby('timestamp').agg({'min', 'max', 'mean'})

Answer 3

If I correctly understand your problem, all you need to do is this:

df.groupby(['provider', df['timestamp'].dt.strftime('%Y-%m-%d')])[['vehicle_id']].nunique()\
  .groupby('timestamp')['vehicle_id'].describe()

In first groupby you'll get the dataframe with with number of unique vehicle_id by provider and day. For provided data sample it is:

                     vehicle_id
provider timestamp             
a        2019-09-11           1
b        2019-09-11           2
c        2019-09-11           1

And in the second it'll be statistics per day. So the result will be

            count      mean      std  min  25%  50%  75%  max
timestamp                                                    
2019-09-11    3.0  1.333333  0.57735  1.0  1.0  1.0  1.5  2.0