pandas: for loop to merge on multiple columns

Question

I have two dataframes which I am trying to combine. The first dataset had agents IDs and multiple columns with skills assigned to each agent.

df1:

Login ID	Skill Assignment 1	Skill Assignment 2	Skill Assignment 3
1000.	500	7008.	1655.
1001.	2590.	3846.	3847.
1002.	500.	2226.	.

The second dataframe has call volume information split by agent and skill.

df2:

Logid.	Skill	Call Volume
1000.	500.	150.
1000.	7008.	54.
1000.	1655.	70.
1001.	2590.	30.
1001.	3846.	240.
1001.	3847.	120.
1002.	500	230.
1002.	2226.	45.

I am trying to add call volume to each agents & skill assignment using for loop, the df has 120 columns, to create new df with skill assignments and call volumes:

df3:

Login ID	Skill Assignment 1	Call Volume 1	Skill Assignment 2	Call Volume 2
1000.	500	150.	7008.	54.
1001.	2590.	30.	3846.	240.
1002.	500.	230.	2226.	45.

I am trying to merge the two datasets and rename the Call Volume column each time is added on:

for i in number_list:
 df3 = pd.merge(df1, df2, how = 'left', left_on = [df1['Login ID'], df1['Skill Assignment ' +str(i)]], right_on = ['Logid', 'Skill'])
 df3.rename({'Call Volume' : 'Call Volume ' + str(i)}, axis = 1, inplace = True).drop(['Logid', 'Skill'], axis = 1)

When the code runs it keeps only the 120 column. Is there a way to keep each column or more adequate way to add the Call Volumes to each skill?

Answer 1

Assuming the '.' character that appears in df1 and df2 is not there,

I think what you're looking for is something along these lines, using melt :

df1_melt=pd.melt(df1,id_vars='Login ID', value_name='Skill', var_name='Skill Assignment')
df3=df2.merge(df1_melt, left_on=['Logid.','Skill'], right_on=['Login ID','Skill'], how='left').drop('Logid.', axis=1)

I believe df3 is the relationship you're looking for. If further grouping is necessary, you can use pivot:

df_pivot=df3.pivot_table(index=['Login ID'], columns=['Skill'], values=['Call Volume','Skill Assignment'] )

---