基于 2 个日期列表分组的 Pandas Groupby 唯一计数

Question

Similar to this question, but adding one more step: Rolling groupby nunique count based on start and end dates

I have a dataframe with a unique ID, a start date, end date, start year, and end year. Over the course of this time, the ID can start, stop, and be restarted.

I would like to get a groupby nunique count of IDs over the course of all year. Currently, I can count unique values for a start and end date of the ID, but how exactly do I incorporate including the years?

fun = pd.DataFrame({'ZIP_KEY': ['A', 'B', 'A'],
                   'start_month': [1, 2, 2],
                   'end_month': [4, 3, 7],
                   'start_year': [2016, 2016, 2017],
                   'end_year': [2016, 2017, 2018]})

fun["month_list"] = fun.apply(lambda x: list(range(x["start_month"], x["end_month"]+1)), axis=1)

fun["year_list"] = fun.apply(lambda x: list(range(x["start_year"], x["end_year"]+1)), axis=1)

fun = fun.explode("month_list")

fun = fun.explode("year_list")

fun.groupby(["year_list", "month_list"])["ZIP_KEY"].nunique()


year_list  month_list
2016       1             1
           2             2
           3             2
           4             1
2017       2             2
           3             2
           4             1
           5             1
           6             1
           7             1
2018       2             1
           3             1
           4             1
           5             1
           6             1
           7             1

If a Zip Key is multi year, my current method is not taking into account full year --> Starts Jan 2018, Ends Feb 2020, then we get [1,2] and [2018,2019,2020] , not the full years for 2018 and 2019. I should get counts [1,2,3,4,5,6,7,8,9,10,11,12] for [2018, 2019] , and [1,2] for 2020

Answer 1

Similar to my other answer, but this time we use pd.date_range with 'MS' frequency instead of range . It's helpful to first create datetime columns that are the first of the month for the provided year-month combinations.

import pandas as pd

# Create start and end datetime column.
for per in ['start', 'end']:
    fun[per] = pd.to_datetime(fun[[f'{per}_year', f'{per}_month']]
                                  .rename(columns={f'{per}_year': 'year', f'{per}_month': 'month'})
                                  .assign(day=1))

df = pd.concat([pd.DataFrame({'date': pd.date_range(st, en, freq='MS'), 'key': k}) 
                for k, st, en in zip(fun['ZIP_KEY'], fun['start'], fun['end'])])

Now group for the output. If you want separate columns:

df.groupby([df.date.dt.year.rename('year'), df.date.dt.month.rename('month')]).key.nunique()

year  month
2016  1        1 # <━┓
      2        2 # <━╋━━┓ 
      3        2 #   A  ┃
      4        2 # <━┛  ┃
      5        1 #      ┃
      6        1 #      ┃
      7        1 #      ┃
      8        1 #      B
      9        1 #      ┃
      10       1 #      ┃
      11       1 #      ┃
      12       1 #      ┃
2017  1        1 #      ┃
      2        2 # <━━━━╋━┓    
      3        2 # <━━━━┛ ┃
      4        1 #        ┃
      5        1 #        ┃
      6        1 #        ┃
      7        1 #        ┃
      8        1 #        ┃
      9        1 #        ┃
      10       1 #        A
      11       1 #        ┃
      12       1 #        ┃
2018  1        1 #        ┃
      2        1 #        ┃
      3        1 #        ┃
      4        1 #        ┃
      5        1 #        ┃
      6        1 #        ┃
      7        1 # <━━━━━━┛

I sometimes prefer grouping by the period:

df.groupby(df.date.dt.to_period('M')).key.nunique()