Pandas: vertical look up with two dataframes

Question

I have a dataframe df1 of coordinate values like this:

    lat         lon         E               N
0   48.010258   -6.156909   90089.518711    -201738.245555
1   48.021648   -6.105887   93961.324059    -200676.766517
2   48.033028   -6.054801   97836.321204    -199614.270439
... ...         ...         ...             ...

and another dataframe df2 that associates a climatic value to each (lat, lon) pair:

    lat         lon        val
0   48.010258   -6.156909  17.11
1   48.021648   -6.105887  22.23
2   48.033028   -6.054801  39.86
... ...         ...        ...

I want to create a new column, df1['corr_pos'] , where each row is given the index of df2 corresponding to the (lat, lon) pair in df1 . It is like using VLOOKUP in Excel, but using two values to identify the correct index instead of using only one. The two values are the coordinate pair.

The output would be:

    lat         lon         E               N               corr_pos
0   48.010258   -6.156909   90089.518711    -201738.245555  0
1   48.021648   -6.105887   93961.324059    -200676.766517  3
2   48.033028   -6.054801   97836.321204    -199614.270439  8
... ...         ...         ...             ...             ...

The dataframes df1 and df2 do not have the same order. How could I implement this in pandas?

Answer 1

I think you need merge with reset_index to create a new column from index :

print (df2)
          lat       lon    val
7   48.010258 -6.156909  17.11
10  48.021648 -6.105887  22.23
12  48.033028 -6.054801  39.86
df = pd.merge(df1, 
              df2.reset_index().drop('val', axis=1).rename(columns={'index':'corr_pos'}), 
              on=['lat','lon'], 
              how='left')
print (df)
         lat       lon             E              N  corr_pos
0  48.010258 -6.156909  90089.518711 -201738.245555         7
1  48.021648 -6.105887  93961.324059 -200676.766517        10
2  48.033028 -6.054801  97836.321204 -199614.270439        12

If df2 has many columns, it is better to use subset as merge will delete them:

df = pd.merge(df1, 
              df2.reset_index()[['lat','lon', 'index']].rename(columns={'index':'corr_pos'}),
              on=['lat','lon'], 
              how='left')
print (df)
         lat       lon             E              N  corr_pos
0  48.010258 -6.156909  90089.518711 -201738.245555         7
1  48.021648 -6.105887  93961.324059 -200676.766517        10
2  48.033028 -6.054801  97836.321204 -199614.270439        12