How to backfill NaN values based on condition that next valid value after NaN is equal to previous valid value

Question

Here is an excerpt from my pandas DataFrame:

df =
TransactionId   Value   AuthenticationId
1214050         8243.12 AE19D686
NaN             8243.12 AE19D686
NaN             8243.12 AE19D686
NaN             8243.12 AE19D686
NaN             8243.12 AE19D686
1214050         8243.12 AE19D686

I want to backfill any NaN values that meet the condition that the current valid (ie not NaN) TransactionId is equal to the previous valid value (first row in the df).

Answer 1

You can check if ffill and bfill are consistent, then apply back-filling conditionally via pd.DataFrame.loc :

bfilled = df['TransactionId'].bfill()

df.loc[df['TransactionId'].ffill() == bfilled, 'TransactionId'] = bfilled