fetch values using reference dataframe from another dataframe and apply calulation

Question

I have two dataframes one dataframes containes four column names as Field_name, field_Type, Unit_Measurement, and Asset Name. and another dataframe contains all the field_name in single row and their corressponding values in the another row. An example

import pandas as pd
df1 = pd.DataFrame({'Field_Name' : ['W_LD(1)', 'R_LD(3)', 'WMEAS_LD(1)', 'WMEAS_LD(2)','W_LN(1)','WMEAS_LN(1)'],
                    'Field_Type' : [est, est, meas, meas,est,meas], 
                    'Unit' : ['mw', 'mv', 'mw', 'mw','mw','mw'], 
                    'Asset_Name' : ['LD(1)', 'LD(3)', 'LD(1)', 'LD(2)','LN(1)','LN(1)']})


Second Dataframe [all the infromation rowwise]
import pandas as pd
df2=pd.Dataframe({['Device_names','W_LD(1)','R_LD(3)','WMEAS_LD(1)','WMEAS_LD(2)','W_LN(1)','WMEAS_LN(1)'],
                  ['Timestamp','2.2','3.3','1.2','3.4','2.3','4.5']})

Now, We have two dataframes. So, basically I have to check if the field type is estimated or measurement, asset_name if LD(1) and Unit is'mw' or 'mv'.

Based on these conditions I have to fetch W_LD(1) and WMEAS_LD(1) from the second dataframe and subtract these two values then the output will be (3.4-2.2)=1.2. this is for one device, similarly i have to do for multiple devices.

Answer 1

if your DataFrame are:

df1 = pd.DataFrame({'Field_Name' : ['W_LD(1)', 'R_LD(3)', 'WMEAS_LD(1)', 'WMEAS_LD(2)','W_LN(1)','WMEAS_LN(1)'],
                    'Field_Type' : ['est', 'est', 'meas', 'meas','est','meas'], 
                    'Unit' : ['mw', 'mv', 'mw', 'mw','mw','mw'], 
                    'Asset_Name' : ['LD(1)', 'LD(3)', 'LD(1)', 'LD(2)','LN(1)','LN(1)']})

df2=pd.DataFrame({'Device_names':['W_LD(1)','R_LD(3)','WMEAS_LD(1)','WMEAS_LD(2)','W_LN(1)','WMEAS_LN(1)'],
                  'Timestamp':['2.2','3.3','1.2','3.4','2.3','4.5']})

1.Solution.

groups=df1.groupby(df1['Asset_Name'])
df1['Timestamp']=df1['Field_Name'].map(df2.set_index('Device_names')['Timestamp'])
df1['Timestamp']=[float(key) for key in df1['Timestamp']]
df1.sort_values('Timestamp',inplace=True)
df1.reset_index(drop=True,inplace=True)
df1['subtract']=groups['Timestamp'].diff()
df1['subtract']=groups['subtract'].transform('first')

---

2.Explanation.

groupby to calculate the subtraction for each group:

groups=df1.groupby(df1['Asset_Name'])

first insert the Timestamp column in df1 in the correct order using Series.map . Order using DataFrame.sort_values ​​to get the subtraction with a positive sign

df1['Timestamp']=df1['Field_Name'].map(df2.set_index('Device_names')['Timestamp'])
df1['Timestamp']=[float(key) for key in df1['Timestamp']]
df1.sort_values('Timestamp',inplace=True)
df1.reset_index(drop=True,inplace=True)

then using groupby.DataFrameGroupBy.diff :

df1['subtract']=groups['Timestamp'].diff()
print(df1)


    Field_Name Field_Type Unit Asset_Name  Timestamp  subtract
0  WMEAS_LD(1)       meas   mw      LD(1)        1.2       NaN
1      W_LD(1)        est   mw      LD(1)        2.2       1.0
2      W_LN(1)        est   mw      LN(1)        2.3       NaN
3      R_LD(3)        est   mv      LD(3)        3.3       NaN
4  WMEAS_LD(2)       meas   mw      LD(2)        3.4       NaN
5  WMEAS_LN(1)       meas   mw      LN(1)        4.5       2.2

Then use transform :

df1['subtract']=groups['subtract'].transform('first')
print(df1)

    Field_Name Field_Type Unit Asset_Name  Timestamp  subtract
0  WMEAS_LD(1)       meas   mw      LD(1)        1.2       1.0
1      W_LD(1)        est   mw      LD(1)        2.2       1.0
2      W_LN(1)        est   mw      LN(1)        2.3       2.2
3      R_LD(3)        est   mv      LD(3)        3.3       NaN
4  WMEAS_LD(2)       meas   mw      LD(2)        3.4       NaN
5  WMEAS_LN(1)       meas   mw      LN(1)        4.5       2.2