How to select matching date/row and previous date(s)/row(s) on Yahoo Finance using Pandas?

Question

My purpose is to select dividend dates and previous date(s) in stock day data df of CSCO stock. I'm able to merge both datasets on index and ticker, however not figured out yet how to select previous date(s)/row(s) as well.

from yahoo_fin.stock_info import get_data, get_dividends
import numpy as np
import pandas as pd

stock_data = get_data('csco', start_date="10/01/2021", end_date="10/07/2021", index_as_date = True, interval="1d") 
    
div_data = get_dividends('csco', "09/01/2021")

stock_data = stock_data.reset_index()
div_data = div_data.reset_index()

print(stock_data)
print(div_data)

Output:

       index       open       high  ...   adjclose    volume  ticker
0 2021-10-01  54.599998  55.410000  ...  54.770000  18338000    CSCO
1 2021-10-04  54.500000  54.680000  ...  54.230000  17084100    CSCO
2 2021-10-05  54.130001  55.029999  ...  54.689999  14135000    CSCO
3 2021-10-06  54.349998  54.380001  ...  53.939999  26339200    CSCO

       index  dividend ticker
0 2021-10-04      0.37   CSCO

Now, I'm able to merge based on index and ticker:

print(pd.merge(stock_data, div_data, on=['index', 'ticker'],  how='inner'))

Output:

       index  open   high        low  ...  adjclose    volume  ticker dividend
0 2021-10-04  54.5  54.68  53.950001  ...     54.23  17084100    CSCO     0.37

However, I want the previous date as well (so the day before the dividend pay date, where no dividends are shared out by the stock. My desired output:

       index  open   high        low  ...  and close    volume  ticker dividend
0 2021-10-01  54.599998  55.410000  ...  54.770000  18338000    CSCO
0 2021-10-04  54.5  54.68  53.950001  ...     54.23  17084100    CSCO     0.37

Any suggestion on how to select previous date(s) as well, in addition to matching row?

Answer 1

Create a boolean mask by checking the dates in the stock_data which are also present in div_data , then remove the rows from stock_data except the rows where the current and previous value in mask is True . Now you can perform the left merge to get the desired result

m = stock_data['index'].isin(div_data['index'])
stock_data[m | m.shift(-1)].merge(div_data, on=['index', 'ticker'], how='left')

---

       index       open   high        low      close  adjclose    volume ticker  dividend
0 2021-10-01  54.599998  55.41  54.040001  55.139999     54.77  18338000   CSCO       NaN
1 2021-10-04  54.500000  54.68  53.950001  54.230000     54.23  17084100   CSCO      0.37