Creating new column in a Pandas df, where each row's value depends on the value of a different column in the row immediately above it

Question

Assume the following Pandas df:

# Import dependency.
import pandas as pd

# Create data for df.
data = {'Value': [1000, 1020, 1011, 1010, 1030, 950, 1001, 1100, 1121, 1131],
        'Dummy_Variable': [0,0,1,0,0,0,1,0,1,1]
       }

# Create DataFrame
df = pd.DataFrame(data)
display(df)

I want to add a new column to the df called 'Placeholder.' The value of Placeholder would be based on the 'Dummy_Variable' column based on the following rules:

• If all previous rows had a 'Dummy_Variable' value of 0, then the 'Placeholder' value for that row would be equal to the 'Value' for that row.
• If the 'Dummy_Variable' value for a row equals 1, then the 'Placeholder' value for that row would be equal to the 'Value' for that row.
• If the 'Dummy_Variable' value for a row equals 0 but the 'Placeholder' value for the row immediately above it is >0, then the 'Placeholder' value for the row would be equal to the 'Placeholder' value for the row immediately above it.

The desired result is a df with new 'Placeholder' column that looks like the df generated by running the code below:

desired_data = {'Value': [1000, 1020, 1011, 1010, 1030, 950, 1001, 1100, 1121, 1131],
        'Dummy_Variable': [0,0,1,0,0,0,1,0,1,1],
        'Placeholder': [1000,1020,1011,1011,1011,1011,1001,1001,1121,1131]}

df1 = pd.DataFrame(desired_data)
display(df1)

I can do this easily in Excel, but I cannot figure out how to do it in Pandas without using a loop. Any help is greatly appreciated. Thanks!

Answer 1

You can use np.where for this:

import pandas as pd
import numpy as np

data = {'Value': [1000, 1020, 1011, 1010, 1030, 950, 1001, 1100, 1121, 1131],
        'Dummy_Variable': [0,0,1,0,0,0,1,0,1,1]
       }

df = pd.DataFrame(data)

df['Placeholder'] = np.where((df.Dummy_Variable.cumsum() == 0) | (df.Dummy_Variable == 1), df.Value, np.nan)

# now forward fill the remaining NaNs
df['Placeholder'].fillna(method='ffill', inplace=True)

df

   Value  Dummy_Variable  Placeholder
0   1000               0       1000.0
1   1020               0       1020.0
2   1011               1       1011.0
3   1010               0       1011.0
4   1030               0       1011.0
5    950               0       1011.0
6   1001               1       1001.0
7   1100               0       1001.0
8   1121               1       1121.0
9   1131               1       1131.0


# check output:
desired_data = {'Value': [1000, 1020, 1011, 1010, 1030, 950, 1001, 1100, 1121, 1131],
        'Dummy_Variable': [0,0,1,0,0,0,1,0,1,1],
        'Placeholder': [1000,1020,1011,1011,1011,1011,1001,1001,1121,1131]}

df1 = pd.DataFrame(desired_data)

check = df['Placeholder'] == df1['Placeholder']
check.sum()==len(df1)
# True