Pandas groupby/value_counts for no values

Question

I have a data set that shows count of loads for each category. Given below is the data I have.

Name,Count1,Count2,PercentDiff,Category
Store A,10,4,0.4,Less than 1%
Store B,20,26,1.3,Less than 5%
Store C,12,48,4,Less than 5%
Store D,30,180,6,Less than 10%

I would like to get the count for each of the below categories

1. Less than 0
2. Less than 1%
3. Less than 5%
4. Less than 10%
5. More than 10%

I have used the below rule to categorise each of these entries:

new.loc[new['PercentDiff'] < 0, 'Category'] = 'Less than 0%'
new.loc[new['PercentDiff'] == 0, 'Category'] = 'Exact match'
new.loc[new['PercentDiff'] < 0.01, 'Category'] = 'Less than 1%'
new.loc[new['PercentDiff'] < 0.05, 'Category'] = 'Less than 5%'
new.loc[new['PercentDiff'] < 0.1, 'Category'] = 'Less than 10%'
new.loc[new['PercentDiff'] == 0, 'Category'] = 'Exact match'
new.loc[new['PercentDiff'] > 0.1, 'Category'] = 'Greater than 10%'
new['PercentDiff1'] = new['PercentDiff'].astype(int)

Output1 = new.groupby(['Category']).agg(lambda x: x.mad())
Output1 = Output1.replace(np.nan, '', regex=True)
SumMail = pd.value_counts(Output1['Category'].values)

However, if the data set has no values for any of the categories I get an error stating no values found for a particular category.

TypeError: 'str' object cannot be interpreted as an integer

KeyError: 'More than 10%'

Could anyone help me modify this code so that it returns 0 for categories that have no records.

Thanks in advance.

Answer 1

You need to define your 'Category' column astype categorical dtype:

df['Category'] = df['Category'].astype('category')
df['Category'] = df['Category'].cat.set_categories(['Less than 0',
                                                    'Less than 1%',
                                                    'Less than 5%',
                                                    'Less than 10%',
                                                    'More than 10%'],
                                                    ordered=True)

df['Category'].value_counts(sort=False)

Output:

Less than 0      0
Less than 1%     1
Less than 5%     2
Less than 10%    1
More than 10%    0
Name: Category, dtype: int64

Answer 2

Check if your dataframe is empty before you do your labeling.

if new['PercentDiff'].empty:
    return 0
else:
    new.loc[new['PercentDiff'] < 0, 'Category'] = 'Less than 0%'
    new.loc[new['PercentDiff'] == 0, 'Category'] = 'Exact match'
    new.loc[new['PercentDiff'] < 0.01, 'Category'] = 'Less than 1%'
    new.loc[new['PercentDiff'] < 0.05, 'Category'] = 'Less than 5%'
    new.loc[new['PercentDiff'] < 0.1, 'Category'] = 'Less than 10%'
    new.loc[new['PercentDiff'] == 0, 'Category'] = 'Exact match'
    new.loc[new['PercentDiff'] > 0.1, 'Category'] = 'Greater than 10%'
    new['PercentDiff1'] = new['PercentDiff'].astype(int)

    Output1 = new.groupby(['Category']).agg(lambda x: x.mad())
    Output1 = Output1.replace(np.nan, '', regex=True)
    SumMail = pd.value_counts(Output1['Category'].values)