pandas groupby 和 map 值列表

Question

There are two dataframes, where 1st dataframe contains list of cells and person names. 2nd dataframe contains the actual values to be mapped to.

df1:

Name           celllist
Bob            ['a', 'v']
April          ['b', 'c']
Amy            ['v']
Linda          ['g', 'r']

df2:

Name    cell    value
Bob      a       4
Bob      g       6
Bob      v       8
Arpil    a       6
Arpil    g       8
Arpil    b       9
Arpil    c       1
Amy      v       2
Amy      b       2

This is what I would like to have, I want to extract values from df2 for the cellist elements and add them back to the df1.

excpected Result: df1:

Name           Group          Group_Name
Bob            ['a', 'v']         [4, 8]       
April          ['b', 'c']         [9, 1]
Amy            ['v']              [2]
Linda          ['g', 'r']         [None, None]

Can someone help me to get this or suggest any better solution?

Answer 1

IIUC, you need to group only those values in the list or NaN, so do:

# create DataFrame to check which values of cell are in Group
res = df2.merge(df1, on='Name', how='right').explode('Group')

# create mask 
mask = res['Group'].eq(res['cell']) | res['cell'].isna()

# filter, group by, agg and rename
output = res[mask].groupby('Name', sort=False).agg({'Group': list, 'value': list}).rename(
    columns={'value': 'Group Name'})
print(output)

Output

        Group  Group Name
Name                     
Bob    [a, v]  [4.0, 8.0]
April  [b, c]  [9.0, 1.0]
Amy       [v]       [2.0]
Linda  [g, r]  [nan, nan]

UPDATE

For keeping the original list order, you could add an additional step:

# create DataFrame to check which values of cell are in Group
res = df1.merge(df2, on='Name', how='left').explode('Group', ignore_index=True)

# reorder DataFrame to keep original list order
res['ord'] = np.arange(len(res))
res['ord'] = res.groupby(['Name', 'Group'])['ord'].transform('first')
res = res.sort_values(by='ord').drop('ord', 1)

# create mask
mask = res['Group'].eq(res['cell']) | res['cell'].isna()

# filter, group by, agg and rename
output = res[mask].groupby('Name', sort=False).agg({'Group': list, 'value': list}).rename(
    columns={'value': 'Group Name'})
print(output)

Answer 2

I believe a few steps in Dani's method above can be truly avoided. The mask creation is unnecessary. If the input dataframe is leveled before merging this solves the problem better and faster. Any keys that don't need matching are ignored in the larger dataframe, and the intermediate results are kept as lean as possible.

# Assuming df as first dataframe and df2 as larger 2nd dataframe
df.explode('celllist').merge(
    df2, how='left',
    left_on=['Name', 'celllist'],
    right_on=['Name', 'cell']
).drop(columns=['cell']).groupby('Name', sort=False).agg(
    {'celllist': list, 'value': list}).rename(
    columns={'celllist': 'Group', 'value': 'Group Name'})

This yields the output just as desired by the OP.

Output:

        Group   Group Name
Name        
Bob     [a, v]  [4.0, 8.0]
April   [b, c]  [9.0, 1.0]
Amy     [v]     [2.0]
Linda   [g, r]  [nan, nan]