如何根据该文件中行中的日期将 CSV 文件在特定日期之间的值传输到另一个 CSV 文件？

Question

Long question: I have two CSV files, one called SF1 which has quarterly data (only 4 times a year) with a datekey column, and one called DAILY which gives data every day. This is financial data so there are ticker columns.

I need to grab the quarterly data for SF1 and write it to the DAILY csv file for all the days that are in between when we get the next quarterly data.

For example, AAPL has quarterly data released in SF1 on 2010-01-01 and its next earnings report is going to be on 2010-03-04. I then need every row in the DAILY file with ticker AAPL between the dates 2010-01-01 until 2010-03-04 to have the same information as that one row on that date in the SF1 file.

So far, I have made a python dictionary that goes through the SF1 file and adds the dates to a list which is the value of the ticker keys in the dictionary. I thought about potentially getting rid of the previous string and just referencing the string that is in the dictionary to go and search for the data to write to the DAILY file.

Some of the columns needed to transfer from the SF1 file to the DAILY file are:

['accoci', 'assets', 'assetsavg', 'assetsc', 'assetsnc', 'assetturnover', 'bvps', 'capex', 'cashneq', 'cashnequsd', 'cor', 'consolinc', 'currentratio', 'de', 'debt', 'debtc', 'debtnc', 'debtusd', 'deferredrev', 'depamor', 'deposits', 'divyield', 'dps', 'ebit']

Code so far:

for ind, row in sf1.iterrows():
    sf1_date = row['datekey']
    sf1_ticker = row['ticker']
    company_date.setdefault(sf1_ticker, []).append(sf1_date)

What would be the best way to solve this problem?

SF1 csv:

ticker,dimension,calendardate,datekey,reportperiod,lastupdated,accoci,assets,assetsavg,assetsc,assetsnc,assetturnover,bvps,capex,cashneq,cashnequsd,cor,consolinc,currentratio,de,debt,debtc,debtnc,debtusd,deferredrev,depamor,deposits,divyield,dps,ebit,ebitda,ebitdamargin,ebitdausd,ebitusd,ebt,eps,epsdil,epsusd,equity,equityavg,equityusd,ev,evebit,evebitda,fcf,fcfps,fxusd,gp,grossmargin,intangibles,intexp,invcap,invcapavg,inventory,investments,investmentsc,investmentsnc,liabilities,liabilitiesc,liabilitiesnc,marketcap,ncf,ncfbus,ncfcommon,ncfdebt,ncfdiv,ncff,ncfi,ncfinv,ncfo,ncfx,netinc,netinccmn,netinccmnusd,netincdis,netincnci,netmargin,opex,opinc,payables,payoutratio,pb,pe,pe1,ppnenet,prefdivis,price,ps,ps1,receivables,retearn,revenue,revenueusd,rnd,roa,roe,roic,ros,sbcomp,sgna,sharefactor,sharesbas,shareswa,shareswadil,sps,tangibles,taxassets,taxexp,taxliabilities,tbvps,workingcapital
A,ARQ,2020-09-14,2020-09-14,2020-09-14,2020-09-14,53000000,7107000000,,4982000000,2125000000,,10.219,-30000000,1368000000,1368000000,1160000000,131000000,2.41,0.584,665000000,111000000,554000000,665000000,281000000,96000000,0,0.0,0.0,202000000,298000000,0.133,298000000,202000000,202000000,0.3,0.3,0.3,4486000000,,4486000000,50960600000,,,354000000,0.806,1.0,1086000000,0.484,0,0,4337000000,,1567000000,42000000,42000000,0,2621000000,2067000000,554000000,51663600000,1368000000,-160000000,2068000000,111000000,0,1192000000,-208000000,-42000000,384000000,0,131000000,131000000,131000000,0,0,0.058,915000000,171000000,635000000,0.0,11.517,,,1408000000,0,114.3,,,1445000000,131000000,2246000000,2246000000,290000000,,,,,0,625000000,1.0,452000000,439000000,440000000,5.116,7107000000,0,71000000,113000000,16.189,2915000000

Daily csv:

ticker,date,lastupdated,ev,evebit,evebitda,marketcap,pb,pe,ps
A,2020-09-14,2020-09-14,31617.1,36.3,26.8,30652.1,6.2,44.4,5.9

Ideal csv after code run (with all the numbers for the assets under them):

ticker,date,lastupdated,ev,evebit,evebitda,marketcap,pb,pe,ps,accoci,assets,assetsavg,assetsc,assetsnc,assetturnover,bvps,capex,cashneq,cashnequsd,cor,consolinc,currentratio,de,debt,debtc,debtnc,debtusd,deferredrev,depamor,deposits,divyield,dps,ebit,ebitda,ebitdamargin,ebitdausd,ebitusd,ebt,eps,epsdil,epsusd,equity,equityavg,equityusd,ev,evebit,evebitda,fcf,fcfps,fxusd,gp,grossmargin,intangibles,intexp,invcap,invcapavg,inventory,investments,investmentsc,investmentsnc,liabilities,liabilitiesc,liabilitiesnc,marketcap,ncf,ncfbus,ncfcommon,ncfdebt,ncfdiv,ncff,ncfi,ncfinv,ncfo,ncfx,netinc,netinccmn,netinccmnusd,netincdis,netincnci,netmargin,opex,opinc,payables,payoutratio,pb,pe,pe1,ppnenet,prefdivis,price,ps,ps1,receivables,retearn,revenue,revenueusd,rnd,roa,roe,roic,ros,sbcomp,sgna,sharefactor,sharesbas,shareswa,shareswadil,sps,tangibles,taxassets,taxexp,taxliabilities,tbvps,workingcapital

Answer 1

The solution is merge_asof it allows to merge date columns to the closer immediately after or before in the second dataframe.

As is it not explicit, I will assume here that daily.date and sf1.datekey are both true date columns, meaning that their dtype is datetime64[ns] . merge_asof cannot use string columns with an object dtype.

I will also assume that you do not want the ev evebit evebitda marketcap pb pe and ps columns from the sf1 dataframes because their names conflict with columns from daily (more on that later):

Code could be:

df = pd.merge_asof(daily, sf1.drop(columns=['dimension', 'calendardate',
                                            'reportperiod','lastupdated',
                                            'ev', 'evebit', 'evebitda',
                                            'marketcap', 'pb', 'pe', 'ps']),
                                   by = 'ticker', left_on='date',
                                   right_on='datekey')

You get the following list of columns: ticker, date, lastupdated, ev, evebit, evebitda, marketcap, pb, pe, ps, datekey, accoci, assets, assetsavg, assetsc, assetsnc, assetturnover, bvps, capex, cashneq, cashnequsd, cor, consolinc, currentratio, de, debt, debtc, debtnc, debtusd, deferredrev, depamor, deposits, divyield, dps, ebit, ebitda, ebitdamargin, ebitdausd, ebitusd, ebt, eps, epsdil, epsusd, equity, equityavg, equityusd, fcf, fcfps, fxusd, gp, grossmargin, intangibles, intexp, invcap, invcapavg, inventory, investments, investmentsc, investmentsnc, liabilities, liabilitiesc, liabilitiesnc, ncf, ncfbus, ncfcommon, ncfdebt, ncfdiv, ncff, ncfi, ncfinv, ncfo, ncfx, netinc, netinccmn, netinccmnusd, netincdis, netincnci, netmargin, opex, opinc, payables, payoutratio, pe1, ppnenet, prefdivis, price, ps1, receivables, retearn, revenue, revenueusd, rnd, roa, roe, roic, ros, sbcomp, sgna, sharefactor, sharesbas, shareswa, shareswadil, sps, tangibles, taxassets, taxexp, taxliabilities, tbvps, workingcapital with their relevant values

---

If you want to keep the columns existing in both dataframe, you will have to rename them. Here is a possible code adding _d to the names of column from daily:

df2 = pd.merge_asof(daily, sf1.drop(columns=['dimension', 'calendardate',
                                            'reportperiod','lastupdated']),
                                   by = 'ticker', left_on='date',
                                   right_on='datekey', suffixes=('_d', ''))

The list of columns is now: ticker, date, lastupdated, ev_d, evebit_d, evebitda_d, marketcap_d, pb_d, pe_d, ps_d, datekey, accoci, ...