从其他数据框分配索引

Question

I have two dataframes. I want to keep all cases in which an observation in dataset B is also in dataset A, but use the index from dataset A.

dfA:

 Index      some_var    some_var2    match_var
   AB          x           y           12
   AC          x           y           13
   AD          x           y           14

dfB:

 Index   Match_var   some_var3    some_var4    
   1       12          z           w           
   2       22          z           w           
   3       14          z           w    

Desired outcome:

 Index      some_var3    some_var4    match_var
   AB          z           w           12
   AD          z           w           14

The problem is that the actual data is too large to perform a merge and dropping unneeded columns and unmatched cases. The memory usage exceeds 100GB RAM.

I wanted to use dfC=dfB.loc[(dfB['Match_var'].isin(dfA['Match_var']))] However, this makes me keep the index of dfB, while I need the one from dfA.

dfA.reset_index(inplace=True)
idx = dfB.loc[(dfB['Match_var'].isin(dfA['Match_var']))]
dfB.loc[idx, 'indexvar'] = dfA['Unnamed']
dfB.set_index(['indexvar'],inplace=True)

also does not work for some reason. the code seems to assign the index from dfA to the wrong observation in the new dataframe

Answer 1

IIUC

pd.concat([dfA.set_index('match_var'),dfB.set_index('Match_var')],join ='inner',axis=1)
Out[782]: 
    Index some_var some_var2  Index some_var3 some_var4
12     AB        x         y      1         z         w
14     AD        x         y      3         z         w

In order to get your output

pd.concat([dfA.set_index('match_var')[['Index']],dfB.set_index('Match_var')[['some_var3','some_var4']]],join ='inner',axis=1).reset_index()
Out[788]: 
   index Index some_var3 some_var4
0     12    AB         z         w
1     14    AD         z         w