DataFrame 按具有字典值的列分组

Question

I'm having a dataframe which contains a column as dictionary. And I need to groupby the column by the dictionary values. For example,

import pandas as pd
    
data = [
         {
            "name":"xx",
            "values":{
                "element":[
                    {
                        "path":"path1/id1"
                    },
                    {
                        "path":"path2/id1"
                    }
                ],
                "nonrequired":[
                    {}
                ]
            }
         },
        {
                "name":"yy",
                "values":{
                    "element":[
                        {
                            "path":"path1/id2"
                        },
                        {
                            "path":"path2/id2"
                        }
                    ],
                    "nonrequired":[
                        {}
                    ]
                }
             }
       ]

df = pd.DataFrame(data)

What I'm looking for,

1. I want to groupby the column "values" by inside specific key.
2. The grouping should be values->element->path
3. The grouping should be based on the partial path values. For example if path="path1/id2", the grouping should be based on path="path1"
4. After grouping I need to extract the result as dictionary.

Expected result:

result = {
            'path1': [
                        {
                            "name":'xx',
                            "renamecolumn":['id1','id2']
                        }
                    ],
            'path2': [
                        {
                            "name":'yy',
                            "renamecolumn":['id1','id2']
                        }
                    ]
        }

Answer 1

Still not 100% sure of the logic of the final dictionary creation as the example input and output don't quite match up. However, here is how you can extract the values and you can create your desired dictionary from there.

# ectract the values and split them on the forward slash
df['split'] = df['values'].apply(lambda x: [item['path'].split('/') for item in x['element']])

# generate the path and ids columns
df['path'] = df['split'].apply(lambda x: [x[i][0] for i in range(0,len(x))])
df['ids'] = df['split'].apply(lambda x: [x[i][1] for i in range(0,len(x))])

# separate out all the lists and 
result = df.drop(['values', 'split'], axis=1) \
  .explode('ids').explode('path').drop_duplicates()

Result is:

  name   path  ids
0   xx  path1  id1
0   xx  path2  id1
1   yy  path1  id2
1   yy  path2  id2