按鍵連接數據框 - 重復數據作為新列

Question

我正面臨下一個情況。 我有兩個數據幀，比如說 df1 和 df2，我需要通過一個鍵（ ID_ed ， ID ）加入它們，第二個數據幀可能有多個鍵出現，我需要的是加入兩個數據幀，並添加鍵作為新列重復出現（如下圖所示）

我嘗試使用merge = df2.join( df1 , lsuffix='_ZID', rsuffix='_IID' , how = "left" )和 concat 操作，但到目前為止沒有運氣。似乎它只保留最后一次出現（如如果它正在覆蓋數據）

非常感謝這方面的任何幫助，並提前致謝。

Answer 1

另一種方法是創建一個將串行計數器ID_ed列， set_index和unstack調用之前pivot_table 。 pivot_table聚合將是first 。 這種方法與this SO answer非常相似

生成數據

import pandas as pd
import numpy as np

a = [['ID_ed','color'],[1,5],[2,8],[3,7]]
b = [['ID','code'],[1,1],[1,5],
    [2,np.nan],[2,20],[2,74],
    [3,10],[3,98],[3,85],
    [3,21],[3,45]]

df1 = pd.DataFrame(a[1:], columns=a[0])
df2 = pd.DataFrame(b[1:], columns=b[0])
print(df1)
   ID_ed  color
0      1      5
1      2      8
2      3      7

print(df2)
   ID  code
0   1   1.0
1   1   5.0
2   2   NaN
3   2  20.0
4   2  74.0
5   3  10.0
6   3  98.0
7   3  85.0
8   3  21.0
9   3  45.0

首先合並和取消堆疊

# Merge and add a serial counter column
df = df1.merge(df2, how='inner', left_on='ID_ed', right_on='ID')
df['counter'] = df.groupby('ID_ed').cumcount()+1
print(df)
   ID_ed  color  ID  code  counter
0      1      5   1   1.0        1
1      1      5   1   5.0        2
2      2      8   2   NaN        1
3      2      8   2  20.0        2
4      2      8   2  74.0        3
5      3      7   3  10.0        1
6      3      7   3  98.0        2
7      3      7   3  85.0        3
8      3      7   3  21.0        4
9      3      7   3  45.0        5

# Set index and unstack
df.set_index(['ID_ed','color','counter']).\
   unstack().\
   swaplevel(1,0,axis=1).\
   sort_index(level=0,axis=1).add_prefix('counter_')
print(df)

counter      counter_1               counter_2             \
            counter_ID counter_code counter_ID counter_code\
ID_ed color                                                \
1     5            1.0          1.0        1.0          5.0\
2     8            2.0          NaN        2.0         20.0\
3     7            3.0         10.0        3.0         98.0 \

 counter_3               counter_4               counter_5             
counter_ID counter_code counter_ID counter_code counter_ID counter_code
       NaN          NaN        NaN          NaN        NaN          NaN
       2.0         74.0        NaN          NaN        NaN          NaN
       3.0         85.0        3.0         21.0        3.0         45.0

接下來生成數據透視表

# Pivot table with 'first' aggregation
dfp = pd.pivot_table(df, index=['ID_ed','color'],
                        columns=['counter'],
                        values=['ID', 'code'],
                        aggfunc='first')
print(dfp)
              ID                      code                        
counter        1    2    3    4    5     1     2     3     4     5
ID_ed color                                                       
1     5      1.0  1.0  NaN  NaN  NaN   1.0   5.0   NaN   NaN   NaN
2     8      2.0  2.0  2.0  NaN  NaN   NaN  20.0  74.0   NaN   NaN
3     7      3.0  3.0  3.0  3.0  3.0  10.0  98.0  85.0  21.0  45.0

最后重命名列並按部分列名切片

# Rename columns
level_1_names = list(dfp.columns.get_level_values(1))
level_0_names = list(dfp.columns.get_level_values(0))
new_cnames = [b+'_'+str(f) for f, b in zip(level_1_names, level_0_names)]
dfp.columns = new_cnames

# Slice by new column names
print(dfp.loc[:, dfp.columns.str.contains('code')].reset_index(drop=False))
   ID_ed  color  code_1  code_2  code_3  code_4  code_5
0      1      5     1.0     5.0     NaN     NaN     NaN
1      2      8     NaN    20.0    74.0     NaN     NaN
2      3      7    10.0    98.0    85.0    21.0    45.0

Answer 2

我會使用 cumcount 和 pivot_table：

In [11]: df1
Out[11]:
   ID  color
0   1      5
1   2      8
2   3      7

In [12]: df2
Out[12]:
   ID  code
0   1   1.0
1   1   5.0
2   2   NaN
3   2  20.0
4   2  74.0

In [13]: res = df1.merge(df2)  # This is a merge if the column names match

In [14]: res
Out[14]:
   ID  color  code
0   1      5   1.0
1   1      5   5.0
2   2      8   NaN
3   2      8  20.0
4   2      8  74.0

In [15]: res['count'] = res.groupby('ID').cumcount()

In [16]: res.pivot_table('code', ['ID', 'color'], 'count')
Out[16]:
count       0     1     2
ID color
1  5      1.0   5.0   NaN
2  8      NaN  20.0  74.0