How can I find values in df_s_t based on values in the rows of df and save the results in df['s_t']?

Question

I have the following DataFrame (df):

print(df.head())
        Date        Contract_Name   Maturity  ...  Call_Put Option_Price         t
0 2016-01-04  Aalberts Industries 2017-10-20  ...         C        12.29  0.049315
1 2016-01-05  Aalberts Industries 2017-10-20  ...         P         0.01  0.049315
2 2016-01-06  Aalberts Industries 2017-10-20  ...         C        11.29  0.049315
3 2016-01-04  WOLTERS-KLUWER      2017-10-20  ...         P         0.01  0.049315
4 2016-01-05  WOLTERS-KLUWER      2017-10-20  ...         C         9.29  0.049315

And I want to add a column df['s_t'] which needs data from df_s_t, this DataFrame looks as follows:

print(df_t_s.head())
        Date  Aalberts Industries  ...  UNILEVER WOLTERS-KLUWER
0 2016-01-04               30.125  ...    38.785         30.150
1 2016-01-05               30.095  ...    39.255         30.425
2 2016-01-06               29.405  ...    38.575         29.920
3 2016-01-07               29.005  ...    37.980         30.690
4 2016-01-08               28.930  ...    37.320         30.070

df['Date'] can be matched with df_s_t['Date'] and df['Contract_Name'] can be matched with the column names of df_s_t.

I hope some one can help me with creating df['s_t'] based on values from df_s_t (as described above). See also an example of df below

print(df.head())
       Date        Contract_Name   Maturity  ...  Call_Put Option_Price         t  s_t
0 2016-01-04  Aalberts Industries 2017-10-20  ...         C        12.29  0.049315 30.125
1 2016-01-05  Aalberts Industries 2017-10-20  ...         P         0.01  0.049315 30.095
2 2016-01-06  Aalberts Industries 2017-10-20  ...         C        11.29  0.049315 29.405
3 2016-01-04  WOLTERS-KLUWER      2017-10-20  ...         P         0.01  0.049315 30.150
4 2016-01-05  WOLTERS-KLUWER      2017-10-20  ...         C         9.29  0.049315 30.425

Solution

df_s_t=pd.melt(df_s_t,id_vars=['Date'])
df_s_t=df_s_t.rename(columns={'variable':"Contract_Name"})
print(df_s_t.head())
        Date        Contract_Name   value
0 2016-01-04  Aalberts Industries  30.125
1 2016-01-05  Aalberts Industries  30.095
2 2016-01-06  Aalberts Industries  29.405
3 2016-01-07  Aalberts Industries  29.005
4 2016-01-08  Aalberts Industries   28.93

Now we can use merge:

df=pd.merge(df,df_s_t,on=['Date','Contract_Name'],how='left')
df=df.rename(columns={'value':'s_t'})
print(df.head())

      Date        Contract_Name   Maturity  ...  Option_Price         t  s_t
0 2017-10-02  Aalberts Industries 2017-10-20  ...         12.29  0.049315  41.29
1 2017-10-02  Aalberts Industries 2017-10-20  ...          0.01  0.049315  41.29
2 2017-10-02  Aalberts Industries 2017-10-20  ...         11.29  0.049315  41.29
3 2017-10-02  Aalberts Industries 2017-10-20  ...          0.01  0.049315  41.29
4 2017-10-02  Aalberts Industries 2017-10-20  ...          9.29  0.049315  41.29

Answer 1

Here is a solution for you.
1) I simplified your data, df1 has only 2 columns (Date and Contract_Name) / df2 has only 4 columns (Date / A / B / C)
2) I melt the df2 (with the variable being called 'Contract_Name') and then groupby Date and Contract_Name
3) I merge both dataframes
4) Print

import pandas as pd
df1 = pd.read_excel('Book1.xlsx', sheet_name='df1')
df2 = pd.melt(pd.read_excel('Book1.xlsx', sheet_name='df2'), id_vars=["Date"],var_name="Contract_Name", value_name="Value").groupby(['Date', 'Contract_Name']).sum().reset_index()
df = pd.merge(df1, df2, how='left', on=['Date','Contract_Name'])
print(df)