Python Map Reduce 从每小时数据中查找每个气象站的每日最高、最低、平均和温度变化

Question

I have a data file for hourly weather data. A screenshot of the data is attached below.

The relevant columns are

1. Wban Number (For each weather station)
2. Dry Bulb Temp

The data is hourly so there are 24 rows for each day and this is for every station. I need to find the daily max, min, mean and variance of "Dry Bulb Temp" from all the weather stations. I can solve this using numpy or other libraries but I am not allowed to use any package that gives statistics. I MUST use MapReduce framework in order to complete this task. I am not familiar with this and can't find help from other questions. How do I approach this? Thanks

Answer 1

Use pandas

import pandas as pd

out = (df.groupby(['WbanNumber', 'YearMonthDay'])['Dry Bulb Temp']
         .agg(['max', 'min', 'mean', 'var']).reset_index())
print(out)

# Output
   WbanNumber YearMonthDay  max  min       mean        var
0        3011     20070101   49   30  38.583333  29.644928
1        3011     20070102   49   30  39.583333  28.514493

My input dataframe:

    WbanNumber YearMonthDay  Dry Bulb Temp
0         3011     20070101             46
1         3011     20070101             34
2         3011     20070101             49
3         3011     20070101             45
4         3011     20070101             30
5         3011     20070101             30
6         3011     20070101             43
7         3011     20070101             40
8         3011     20070101             37
9         3011     20070101             40
10        3011     20070101             44
11        3011     20070101             36
12        3011     20070101             45
13        3011     20070101             40
14        3011     20070101             30
15        3011     20070101             34
16        3011     20070101             41
17        3011     20070101             33
18        3011     20070101             43
19        3011     20070101             40
20        3011     20070101             39
21        3011     20070101             32
22        3011     20070101             36
23        3011     20070101             39
24        3011     20070102             32
25        3011     20070102             39
26        3011     20070102             47
27        3011     20070102             34
28        3011     20070102             30
29        3011     20070102             40
30        3011     20070102             48
31        3011     20070102             35
32        3011     20070102             34
33        3011     20070102             40
34        3011     20070102             42
35        3011     20070102             38
36        3011     20070102             36
37        3011     20070102             32
38        3011     20070102             40
39        3011     20070102             41
40        3011     20070102             38
41        3011     20070102             38
42        3011     20070102             45
43        3011     20070102             39
44        3011     20070102             48
45        3011     20070102             44
46        3011     20070102             41
47        3011     20070102             49