如何按时间值将CSV中的数据帧分组，以便可以计算每小时的频率？

Question

Here is my dataset dataset_for_this_Question

I want to group dataset according to 'Time' and 'Type', So that I can get frequency of 'Name' for each hourly basis. [Per Hour How many Types and what are their Names]. My first requirement is to group dataset according to 'Time' - Hourly basis.

I am using Pandas in Python.

Answer 1

You can groupby the first 13 characters of your Time column and Type , and then just use value_counts , or group by all three and use .size .

df.groupby([df.Time.str[0:13], 'Type']).Name.value_counts()
# or
df.groupby([df.Time.str[0:13], 'Type', 'Name']).size()

Outputs:

Time           Type                      Name                               
2018-04-07 15  COMMUNICATIONS ALARM      Device Management IP is Unreachable    141
2018-04-07 16  COMMUNICATIONS ALARM      Device Management IP is Unreachable     64
2018-04-07 17  COMMUNICATIONS ALARM      Device Management IP is Unreachable      6
...
2018-04-09 14  COMMUNICATIONS ALARM      Device Management IP is Unreachable      8
2018-04-09 15  COMMUNICATIONS ALARM      Device Management IP is Unreachable     11
2018-04-09 16  COMMUNICATIONS ALARM      Device Management IP is Unreachable      5
2018-04-09 17  QUALITY_OF_SERVICE_ALARM  Temperature Absolute High               64
                                         Memory Absolute High                     1

---

Given your data format, slicing by the string characters is perfectly fine, but perhaps case specific. In general, you can convert your Time column to a datetime object which gives you access to a lot of additional functionality. In this case, you can floor to the nearest hour.

df['Time'] = pd.to_datetime(df.Time)
df.groupby([df.Time.dt.floor('1H'), 'Type', 'Name']).size()

Will yield:

Time                 Type                      Name                               
2018-04-07 15:00:00  COMMUNICATIONS ALARM      Device Management IP is Unreachable    141
2018-04-07 16:00:00  COMMUNICATIONS ALARM      Device Management IP is Unreachable     64
2018-04-07 17:00:00  COMMUNICATIONS ALARM      Device Management IP is Unreachable      6
2018-04-07 18:00:00  COMMUNICATIONS ALARM      Device Management IP is Unreachable      7
...

Answer 2

Create time format for each hour and then collect data for each hour and then add each 2 hours of data

dates = pd.date_range(start='2018-04-09', end='2018-05-17', freq='H')
dates

Then you will get these output:

DatetimeIndex(['2018-04-09 00:00:00', '2018-04-09 01:00:00',
           '2018-04-09 02:00:00', '2018-04-09 03:00:00',
           '2018-04-09 04:00:00', '2018-04-09 05:00:00',
           '2018-04-09 06:00:00', '2018-04-09 07:00:00',
           '2018-04-09 08:00:00', '2018-04-09 09:00:00',
           ...
           '2018-05-16 15:00:00', '2018-05-16 16:00:00',
           '2018-05-16 17:00:00', '2018-05-16 18:00:00',
           '2018-05-16 19:00:00', '2018-05-16 20:00:00',
           '2018-05-16 21:00:00', '2018-05-16 22:00:00',
           '2018-05-16 23:00:00', '2018-05-17 00:00:00'],
          dtype='datetime64[ns]', length=913, freq='H')

df_new = pd.DataFrame()   

This dataframe is to collect each hour of data from main dataframe [df]

for x in range(0, len(dates) - 2, 2):
    start_date = str(dates[x])[:13]
    end_date = str(dates[x+1])[:13]
    print(start_date, end_date)


df_temp = df[start_date:end_date]   
# Each hour of data collected to new dataframe. 

After getting data in dataframe, we can do a lot of operations.