Pandas df to multiple nested dictionary/json

Question

I am having trouble converting a df to a 3 level nested dictionary, is there any way to do this without an ugly function that loops over each row? Something along the lines of .groupby .apply?

Input/DF:

project,stage,error_code,count
Project_1,stage_1,0,8
Project_1,stage_1,1103,3
Project_1,stage_2,0,4
Project_1,stage_2,1103,2
Project_2,stage_1,0,14
Project_2,stage_1,1103,2
Project_2,stage_1,1105,1
Project_2,stage_2,0,5

Desired output:

[
    'Project_1': {
        'stage_1': {
            '0': 8,
            '1103': 3
        },
        'stage_2': {
            '0': 14,
            '1103': 2
        }
    },
    'Project_2': {
        'stage_1': {
            '0': 14,
            '1103': 2,
            '1105': 1
        },
        'stage_2': {
            '0': 5,
        }
    }
]

Answer 1

You can do with groupby and unstack

d=df.groupby(['project','stage']).\
        apply(lambda x : dict(zip(x['error_code'],x['count']))).\
           unstack(0).to_dict()
Out[12]: 
{'Project_1': {'stage_1': {0: 8, 1103: 3}, 'stage_2': {0: 4, 1103: 2}},
 'Project_2': {'stage_1': {0: 14, 1103: 2, 1105: 1}, 'stage_2': {0: 5}}}