Pandas Aggregate groupby

Question

I have a dataframe that looks conceptually like the following:

df = pd.DataFrame({
    "a": [1, 1, 1, 2, 2,3],
    "b": ["a", "a", "c", "a", "d","a"],
    "c": ["2", "3", "4", "2", "3","2"]
})

      a    b    c
  0   1   'a'  '2' 
  1   1   'a'  '3'
  2   1   'c'  '4'
  3   2   'a'  '2'
  4   2   'd'  '3'
  5   3   'a'  '2'

For each group in a I need to count the unique (b,c) values up to here.

So in this example the ouptut should be [3,4,4] .

(Because in group 1 there are 3 unique (b,c) pairs, and in group 1 and 2 together there are 4 unique (b,c) values, and in group 1 and 2 and 3 together there are also only 4 unique (b,c) values.

I tried using expanding with groupby and nunique but I couldn't figure out the syntax.

Any help will be appreciated!

Answer 1

First find the indices of the unique rows:

idx = df[['b','c']].drop_duplicates().index

Then find the cumulative sum of the number of rows left in each group:

np.cumsum(df.iloc[idx,:].groupby('a').count()['b'])

returning

a
1    3
2    4

Answer 2

I improved Dan's answer.

df['t'] = np.cumsum(~df[['b','c']].duplicated())
df.groupby('a')['t'].last()
Out[44]: 
a
1    3
2    4
3    4
Name: t, dtype: int64

Answer 3

This is a tricky question. Is this what you are after?

result = (
    df.a.drop_duplicates(keep='last')
    .reset_index()['index']
    .apply(lambda x: df.loc[df.index<=x].pipe(lambda x: (x.b+x.c).nunique()))
     )


result
Out[27]: 
0    3
1    4
Name: index, dtype: int64

Answer 4

You can use the drop_duplicates after your groupby and get the shape of the object :

df = pd.DataFrame({
    "a": [1, 1, 1, 2, 2],
    "b": ["a", "a", "c", "a", "d"],
    "c": ["2", "3", "4", "2", "3"]
})
result = df.groupby("a").apply(lambda x: x.drop_duplicates().shape[0])

If you want to convert the result in list after :

result.tolist()

The result will be [3,2] with your example because you have 3 unique couples for group a=1 and 2 unique couples for group a=2 .

If you want the number of unique couple for colums 'b' and 'c' :

df[["b", "c"]].drop_duplicates().shape[0]