How to perform a multiple groupby and transform count with a condition in pandas

Question

This is an extension of the question here: here

I am trying add an extra column to the grouby:

# Import pandas library 
import pandas as pd
import numpy as np

# data
data = [['tom', 10,2,'c',100,'x'], ['tom',16 ,3,'a',100,'x'], ['tom', 22,2,'a',100,'x'],
        ['matt', 10,1,'c',100,'x'], ['matt', 15,5,'b',100,'x'], ['matt', 14,1,'b',100,'x']]

# Create the pandas DataFrame 
df = pd.DataFrame(data, columns = ['Name', 'Attempts','Score','Category','Rating','Other'])
df['AttemptsbyRating'] = df.groupby(by=['Rating','Other'])['Attempts'].transform('count')
df

Then i try to add another column for the sum of rows that have a Score greater than 1 (which should equal 4):

df['scoregreaterthan1'] = df['Score'].gt(1).groupby(by=df[['Rating','Other']]).transform('sum')

But i am getting a

ValueError: Grouper for '<class 'pandas.core.frame.DataFrame'>' not 1-dimensional

Any ideas? thanks very much!

Answer 1

df['Score'].gt(1) is returning a boolean series rather than a dataframe. You need to return a dataframe first before you can groupby the relevant columns.

use:

df = df[df['Score'].gt(1)]
df['scoregreaterthan1'] = df.groupby(['Rating','Other'])['Score'].transform('count')
df

output:

    Name    Attempts    Score   Category    Rating  Other   AttemptsbyRating    scoregreaterthan1
0   tom     10          2       c           100     x       6                4
1   tom     16          3       a           100     x       6                4
2   tom     22          2       a           100     x       6                4
4   matt    15          5       b           100     x       6                4

If you want to keep the people who have a score that is not greater than one, then instead of this:

df = df[df['Score'].gt(1)]
df['scoregreaterthan1'] = df.groupby(['Rating','Other'])['Score'].transform('count')

do this:

df['scoregreaterthan1'] = df[df['Score'].gt(1)].groupby(['Rating','Other'])['Score'].transform('count')
df['scoregreaterthan1'] = df['scoregreaterthan1'].ffill().astype(int)

output 2:

    Name    Attempts    Score   Category    Rating  Other   AttemptsbyRating    scoregreaterthan1
0   tom     10  2   c   100 x   6   4
1   tom     16  3   a   100 x   6   4
2   tom     22  2   a   100 x   6   4
3   matt    10  1   c   100 x   6   4
4   matt    15  5   b   100 x   6   4
5   matt    14  1   b   100 x   6   4