Compare two dataframes and choose the one based on criteria

Question

So a bit complicated example, at least for me. I have two dataframes. They have different interval lengths and I have a few criteria in order to merge them. I just don't know how to go about it. There will be multiple HoleIDs. Issue is overlapping intervals between both files and want the script to break it down if there is multiple values in each that it can break down to.

File1

HoleID,From,To,Value
DH-1,0,15,-99
DH-1,15,25,0.01
DH-1,25,100,0.022

File2

HoleID,From,To,Value
DH-1,0,5,0.08
DH-1,5,10,1
DH-1,10,25,0.04
DH-1,25,50,-99
DH-1,50,100,0.066

Want

HoleID,From,To,Value
DH-1,0,5,0.08
DH-1,5,10,1
DH-1,10,25,0.04
DH-1,25,50,0.022
DH-1,50,100,0.066

-99 = missing in this instance

1. File2 to take precendence over File1 when File1 has -99 as the value
2. If File1 to take precendence over File2 when File2 has -99
3. Merge/overwrite based off holeid (since there will be more than just one) and From/To intervals.
4. If a value exists for both File1 and File2, Let the Value in File 2 be chosen

Merge doesn't do it.

Answer 1

You can merge the two frames on HoleID and From and conditionally update Value if equal to -99 with mask :

df3 = df2.merge(df1.drop('To', axis=1), on=['HoleID', 'From'], how='left', suffixes=['', '_y'])
df3['Value'] = df3['Value'].mask(df3['Value'] == -99, df3['Value_y'])
df3 = df3.drop('Value_y', axis=1)
df3
Out[1]: 
  HoleID  From   To  Value
0   DH-1     0    5  0.080
1   DH-1     5   10  1.000
2   DH-1    10   25  0.040
3   DH-1    25   50  0.022
4   DH-1    50  100  0.066