Using pandas I need to create a new column that takes a value from a previous row

Question

I have many rows of data and one of the columns is a flag. I have 3 identifiers that need to match between rows.

What I have:

part number, datetime1, previousdatetime1, datetime2, previousdatetime2, flag

What I need:

partnumber, datetime1, previousdatetime1, datetime2, previousdatetime2, flag, previous_flag

I need to find flag from the row where partnumber matches, and where the previousdatetime1(current row*) == datetime1(other row)*, and the previousdatetime2(current row) == datetime2(other row).

*To note, the rows are not necessarily in order so the previous row may come later in the dataframe

I'm not quite sure where to start. I got this logic working in PBI using a LookUpValue and basically finding where partnumber = Value(partnumber), datetime1 = Value(datetime1), datetime2 = Value(datetime2). Thanks for the help!

Answer 1

Okay, so assuming you've read this in as a pandas dataframe df1:

(1) Make a copy of the dataframe:

df2=df1.copy()

(2) For sanity, drop some columns in df2

df2.drop(['previousdatetime1','previousdatetime2'],axis=1,inplace=True) 

Now you have a df2 that has columns:

['partnumber','datetime1','datetime2','flag']

(3) Merge the two dataframes

newdf=df1.merge(df2,how='left',left_on=['partnumber','previousdatetime1'],right_on=['partnumber','datetime1'],suffixes=('','_previous')) 

Now you have a newdf that has columns:

['partnumber','datetime1','previousdatetime1','datetime2','previousdatetime2','flag','partnumber_previous','datetime1_previous','datetime2_previous','flag_previous']

(4) Drop the unnecessary columns

newdf.drop(['partnumber_previous', 'datetime1_previous', 'datetime2_previous'],axis=1,inplace=True)

Now you have a newdf that has columns:

['partnumber','datetime1','previousdatetime1','datetime2','previousdatetime2','flag','flag_previous']