比较和移动 Pandas DataFrame 列中的值 - Python

Question

I have the following pandas DataFrame populated:

The ids contained in the first cell of the column match the ids in the three cells of the second column. The contents of each cell aren't fixed (so those aren't literal string values, but data fetched from variable json api output).

How would I go about comparing the contents of both columns (and since the contents aren't fixed, I suppose this would have to be done variably rather than by literal strings), and if there's a match, move the matches to the corresponding cell next to it? Hope that makes sense, this is the type of output I'm looking for:

Answer 1

# sample data
df = pd.DataFrame(np.array([{'data': [{'id': '12345', 'type': 'education'}, {'id': '23456', 'type': 'education'}, {'id': '34567', 'type': 'education'}]},
                            {'data': [{'id': '45678', 'type': 'education'}, {'id': '56789', 'type': 'education'}]},
                            {'data': [{'id': '78999', 'type': 'education'}]}]), columns=['Edu ID'])

# create a new frame but orient the index and explode
df_e = pd.DataFrame.from_dict(df['Edu ID'].to_dict(), orient='index')['data'].explode()
# take the new frame and convert it to a list then groupby the index and create a list of ids
final_df = df.join(pd.DataFrame(df_e.tolist(), index=df_e.index).groupby(level=0)['id'].agg(list))


                                              Edu ID                     id
0  {'data': [{'id': '12345', 'type': 'education'}...  [12345, 23456, 34567]
1  {'data': [{'id': '45678', 'type': 'education'}...         [45678, 56789]
2   {'data': [{'id': '78999', 'type': 'education'}]}                [78999]

If you need to filter where type == education then

# create a new frame but orient the index and explode
df_e = pd.DataFrame.from_dict(df['Edu ID'].to_dict(), orient='index')['data'].explode()

# take the new frame and convert it to a list and create a new frame
df_edu = pd.DataFrame(df_e.tolist(), index=df_e.index)

# use join but filter type to equal education and then gorupby and convert ids to a list
final_df = df.join(df_edu[df_edu['type'] == 'education'].groupby(level=0)['id'].agg(list))

Answer 2

Approach - expand embedded dict and list to dataframe rows and columns. Then build CSV of IDs.

df = pd.DataFrame([
    {"Edu ID":{"data":[
    {"id":1,"type":"educations"},
    {"id":2,"type":"educations"},
    {"id":3,"type":"educations"},
                                ]}, "Education ID":1},
    {"Edu ID":{"data":[
    {"id":4,"type":"educations"},
                                ]}, "Education ID":2},
             ])

(
# convert dict to columns, then explode "data" list,  then convert dicts in list to columns
pd.json_normalize(pd.json_normalize(df.to_dict(orient="records"))
                  .reset_index()
                  .explode("Edu ID.data")
                  .to_dict(orient="records"))
    # build required CSV from embedded dicts
    .groupby("index")["Edu ID.data.id"].agg(lambda x: ",".join(list(x.astype(str))))
    .to_frame()
    # bring it together with original DF
    .join(df)
    .rename(columns={"Edu ID.data.id":"Education ID", "Education ID":"OLD Education ID"})
)

output

Education ID                                                                                                        Edu ID  OLD Education ID
       1,2,3 {'data': [{'id': 1, 'type': 'educations'}, {'id': 2, 'type': 'educations'}, {'id': 3, 'type': 'educations'}]}                 1
           4                                                                   {'data': [{'id': 4, 'type': 'educations'}]}                 2