join data frames on rows and columns

Question

 df
            val      ts   user   visit_id    
id                                         
1             1       1    x          0.0  
21           21       1    z          0.0  
71           71       1    y          0.0  
2             2       2    x          0.0  
22           22       2    z          0.0  
72           72       2    y          0.0  
3             3       3    x          0.0  
23           23       3    z          0.0  
73           73       3    y          0.0  
4             4       4    x          0.0  
24           24       4    z          0.0  
74           74       4    y          0.0  
60           60      60    x          1.0  
90           90      60    z          1.0  
80           80      60    y          1.0  
91           91      61    z          1.0  
81           81      61    y          1.0  
61           61      61    x          1.0  
82           82      62    y          1.0  
92           92      62    z          1.0  
62           62      62    x          1.0  
83           83      63    y          1.0  
93           93      63    z          1.0  
63           63      63    x          1.0  
64           64      64    z          1.0  
64           64      64    x          1.0  
94           94      64    y          1.0  
160         160     150    y          2.0  
180         180     150    z          2.0  
150         150     150    x          2.0  
185         185     155    z          2.0  
155         155     155    x          2.0  
165         165     155    y          2.0  
156         156     156    y          2.0  
166         166     163    x          2.0  
186         186     183    z          2.0  

and

  df2
    visit_id    0.0     1.0     2.0
    user            
    x           0.0     28.0    56.0
    y           0.0     28.0    56.0
    z           0.0     28.0    56.0

How to merge df2 into df such that all the rows of df where user and visit_id are equal to user and visit id from df2 get that corresponding cell from df2.

for example

all rows of df.loc[(df.user == 'x') & (df.visit_id == 0), 'val'] should get the value that is top left cell of df2

Answer 1

I think need for inner join reshape df2 by stack or melt first and then merge :

df = df.merge(df2.stack().reset_index(name='val1'), on=['user','visit_id'], how='left')

Alternative:

df = df.merge(df2.reset_index().melt('user', value_name='val1'), on=['user','visit_id'])

Or for left join use MultiIndex :

print (df.tail())
     val   ts user  visit_id
id                          
155  155  155    x       2.0
165  165  155    y       2.0
156  156  156   y1       2.0 <-change values of user for not match
166  166  163   x1       2.0 <-change values of user for not match
186  186  183   z1       2.0 <-change values of user for not match

#join together
df = df.join(df2.stack().rename('val'), on=['user','visit_id'], lsuffix='_')
#replace val column if match, else get values of original
df['val'] = df['val'].combine_first(df['val_'])
#remove original column val
df = df.drop('val_', axis=1)
print (df)
      ts user  visit_id    val
id                            
1      1    x       0.0    0.0
21     1    z       0.0    0.0
71     1    y       0.0    0.0
2      2    x       0.0    0.0
22     2    z       0.0    0.0
.
.
.
160  150    y       2.0   56.0
180  150    z       2.0   56.0
150  150    x       2.0   56.0
185  155    z       2.0   56.0
155  155    x       2.0   56.0
165  155    y       2.0   56.0
156  156   y1       2.0  156.0
166  163   x1       2.0  166.0
186  183   z1       2.0  186.0