column with previous value "Month"

Question

I have this example of df

I do some transformations on it and I need to get my mark value from the previous month in a new column to make comparisons

import pandas as pd
from pandas.tseries.offsets import BMonthEnd



df = pd.DataFrame({'Found':['A','A','A','A','A','B','B','B','B'],
   'Date':['14/10/2021','19/10/2021','29/10/2021','30/09/2021','20/09/2021','20/10/2021','29/10/2021','15/09/2021','30/09/2021'],
   'Mark':[1,2,3,4,3,1,2,5,8]

  })

df['Date'] = pd.to_datetime(df['Date'])
df['Date'] = pd.to_datetime(df['Date'], format = '%Y/%m/%d')
df['LastDay'] = pd.to_datetime(df['Date']) - BMonthEnd(0)

I'm using group by to get the highest value for each month and create a new column with that value but I can also use a lambda function.

mark_last_day = df.loc[df.apply(lambda x: x['Date']==x['LastDay'], 1)]

or

𝚖𝚊𝚛𝚔_𝚕𝚊𝚜𝚝_𝚍𝚊𝚢 = 𝚍𝚏.𝚕𝚘𝚌[𝚍𝚏.𝚐𝚛𝚘𝚞𝚙𝚋𝚢(['Found', 'LastDay'])['Mark'].𝚒𝚍𝚡max()]

df.merge(mark_last_day[['Found', 'LastDay', 'Mark']],
     on=['Found', 'LastDay'],
     how='left', suffixes=['', '_LastDay'])

resulting in this

how can I create a new column with the result of the previous month as in this example I filled in manually

Answer 1

If months are consecutive like in sample data is possible use DataFrameGroupBy.shift :

df['Date'] = pd.to_datetime(df['Date'], format = '%Y/%m/%d')
df['LastDay'] = pd.to_datetime(df['Date']) - BMonthEnd(0)

s1  = df.set_index('Mark').groupby(['Found', 'LastDay'])['Date'].idxmax()
s2 = s1.groupby(level=0).shift()

df = df.join(pd.concat([s1.rename('Mark_LastDay'), 
                        s2.rename('Mark_LastDayPrevMonth')], axis=1), 
             on=['Found', 'LastDay'])
print (df)
  Found       Date  Mark    LastDay  Mark_LastDay  Mark_LastDayPrevMonth
0     A 2021-10-14     1 2021-10-29             3                    4.0
1     A 2021-10-19     2 2021-10-29             3                    4.0
2     A 2021-10-29     3 2021-10-29             3                    4.0
3     A 2021-09-30     4 2021-09-30             4                    NaN
4     A 2021-09-20     3 2021-09-30             4                    NaN
5     B 2021-10-20     1 2021-10-29             2                    8.0
6     B 2021-10-29     2 2021-10-29             2                    8.0
7     B 2021-09-15     5 2021-09-30             8                    NaN
8     B 2021-09-30     8 2021-09-30             8                    NaN