从 pandas.core.groupby.generic.DataFrameGroupBy object 获取值

Question

Hi So I have dataframe like this, which has 71 unique values in time column, 721 unique values in lat column, 1440 unique values in lon column and all the values in temp column are unique.

Dataframe Sample:

  time        latitude  longitude       temp
1950-01-01      90.0     0.00         49654.792969
1950-01-01      90.0     0.25         49654.792969
   .              .       .                .
   .              .       .                .
73715040 rows * 4 cloumn

Now I want to groupby using lat and lon column to get all the values of temp across all time period for all grid or pairs which will have 1038240 rows(721 lat*1440 lon) , so I'm doing like this.

df = df.groupby(['latitude', 'longitude'])

Now as it is pandas.core.groupby.generic.DataFrameGroupBy object I'm not able to access the values from it. So I'm trying to convert it into dataframe by df.apply(pd.DataFrame) but this is taking lot of time and my kernel is getting crash. So is there any other way to get the records, or am I doing something wrong here. Please suggest alternate way if possible.

Answer 1

The object type pandas.core.groupby.generic.DataFrameGroupBy is a list of tuples, where the first element is the groupby element and the second the dataframe for that group.

See the example below:

Creating test dataframe

import pandas as pd

df = pd.DataFrame({"ColA": [1,1,1,2,2,3,3,3], "ColB": [5,5,6,7,7,8,8,9], "ColC": [1,2,3,4,5,6,7,8]})

The test dataframe

>>> df
   ColA  ColB  ColC
0     1     5     1
1     1     5     2
2     1     6     3
3     2     7     4
4     2     7     5
5     3     8     6
6     3     8     7
7     3     9     8

Grouping dataframe

>>> groups = df.groupby(["ColA", "ColB"])

>>> type(groups)
<class 'pandas.core.groupby.generic.DataFrameGroupBy'>

Showing results

>>> for group in groups:
...     g, value = group
...     print(f"Key = {g}")
...     print(value)
...     print(80*"-")
...
Key = (1, 5)
   ColA  ColB  ColC
0     1     5     1
1     1     5     2
--------------------------------------------------------------------------------
Key = (1, 6)
   ColA  ColB  ColC
2     1     6     3
--------------------------------------------------------------------------------
Key = (2, 7)
   ColA  ColB  ColC
3     2     7     4
4     2     7     5
--------------------------------------------------------------------------------
Key = (3, 8)
   ColA  ColB  ColC
5     3     8     6
6     3     8     7
--------------------------------------------------------------------------------

IMPORTANT

As commented by @HenriChab, using aggregate or, for example, sum will return a dataframe type not a group type

>>> new_df = df.groupby(["ColA", "ColB"]).sum()
>>> new_df
           ColC
ColA ColB
1    5        3
     6        3
2    7        9
3    8       13
     9        8

Finally you can reset the index.

>>> new_df.reset_index(inplace=True)

>>> new_df
   ColA  ColB  ColC
0     1     5     3
1     1     6     3
2     2     7     9
3     3     8    13
4     3     9     8

Answer 2

This should work for you:

df.groupby(['latitude', 'longitude']).aggregate(lambda x: ','.join(map(str, x)))