How do I group a time series by hour of day?

Question

I have a time series and I want to group the rows by hour of day (regardless of date) and visualize these as boxplots. So I'd want 24 boxplots starting from hour 1, then hour 2, then hour 3 and so on.

The way I see this working is splitting the dataset up into 24 series (1 for each hour of the day), creating a boxplot for each series and then plotting this on the same axes.

The only way I can think of to do this is to manually select all the values between each hour, is there a faster way?

some sample data:

Date    Actual Consumption
2018-01-01 00:00:00 47.05
2018-01-01 00:15:00 46
2018-01-01 00:30:00 44
2018-01-01 00:45:00 45
2018-01-01 01:00:00 43.5
2018-01-01 01:15:00 43.5
2018-01-01 01:30:00 43
2018-01-01 01:45:00 42.5
2018-01-01 02:00:00 43
2018-01-01 02:15:00 42.5
2018-01-01 02:30:00 41
2018-01-01 02:45:00 42.5
2018-01-01 03:00:00 42.04
2018-01-01 03:15:00 41.96
2018-01-01 03:30:00 44
2018-01-01 03:45:00 44
2018-01-01 04:00:00 43.54
2018-01-01 04:15:00 43.46
2018-01-01 04:30:00 43.5
2018-01-01 04:45:00 43
2018-01-01 05:00:00 42.04

This is what i've tried so far:

zero = df.between_time('00:00', '00:59')
one = df.between_time('01:00', '01:59')
two = df.between_time('02:00', '02:59')

and then I would plot a boxplot for each of these on the same axes. However it's very tedious to do this for all 24 hours in a day.

This is the kind of output I want: https://www.researchgate.net/figure/Boxplot-of-the-NOx-data-by-hour-of-the-day_fig1_24054015

Answer 1

there are 2 steps to achieve this:

1. convert Actual to date time:

    df.Actual = pd.to_datetime(df.Actual) 

2. Group by the hour:

    df.groupby([df.Date, df.Actual.dt.hour+1]).Consumption.sum().reset_index() 

I assumed you wanted to sum the Consumption (unless you wish to have mean or whatever just change it). One note: hour+1 so it will start from 1 and not 0 (remove it if you wish 0 to be midnight).

desired result:

    Date    Actual  Consumption
0   2018-01-01  1   182.05
1   2018-01-01  2   172.50
2   2018-01-01  3   169.00
3   2018-01-01  4   172.00
4   2018-01-01  5   173.50
5   2018-01-01  6   42.04