在具有不同日期的多列上合并或联接两个数据框

Question

I am trying to combine these two dataframes (df1 and df2):

gmDate n pf pa 0 2012-10-31 ATL 0 0 1 2012-10-31 BKN 0 0 2 2012-10-31 BOS 107 120 3 2012-10-31 CHA 0 0 4 2012-10-31 CHI 0 0 5 2012-10-31 CLE 94 84 6 2012-10-31 DAL 99 91 7 2012-10-31 DEN 0 0 8 2012-10-31 DET 0 0 9 2012-10-31 GS 0 0

gmDate t tw tf ta o ow of oa 0 2012-10-30 WAS 0 0 0 CLE 1 0 0 1 2012-10-30 BOS 0 0 0 MIA 1 0 0 2 2012-10-30 DAL 1 0 0 LAL 0 0 0 3 2012-10-31 DEN 0 0 0 PHI 1 0 0 4 2012-10-31 IND 1 0 0 TOR 0 0 0 5 2012-10-31 HOU 1 0 0 DET 0 0 0 6 2012-10-31 SAC 0 0 0 CHI 1 0 0 7 2012-10-31 SA 1 0 0 NO 0 0 0 8 2012-10-31 DAL 0 0 0 UTA 1 0 0 9 2012-10-31 GS 1 0 0 PHO 0 0 0

I need pf and pa in df1 to populate into tf and ta or of and oa in df2 based on matching gmDate and n against t or o in df2. The df1 includes every day in the calendar, whether or not a team played that day, and df2 contains only the days a team played. I have not been able to get a merge or join to work for me.

Currently I have been trying to do this by running two separate for loops:

for s in range(0, len(df1)): for d in range(0, len(df2): if df1.iloc[s,0] == df2.iloc[d,0] and df1.iloc[s,1] == df2.iloc[d,1]: df2.iloc[d,3] = df1.iloc[s,2] df2.iloc[d,4] = df1.iloc[s,3]

and then:

for s in range(0, len(df1)): for d in range(0, len(df2): if df1.iloc[s,0] == df2.iloc[d,0] and df1.iloc[s,1] == df2.iloc[d,5]: df2.iloc[d,7] = df1.iloc[s,2] df2.iloc[d,8] = df1.iloc[s,3]

Each of them takes a VERY long time to run. df1 has a length of 29,520 and df2 has a length of 7,379.

Sorry if this is too confusing. I looking either the best way to do this with a merge/join or not have my loops run forever.

Thank you in advance for your help.

Answer 1

I think I have understood well what you want. My idea is that you can do this:

you want to substitute in the columns tf and ta of the dataframe df2 with the columns of the dataframe df1 , pf and pa when the dates coincide that is: (df1 ['gmDate']. values) == (df2 ['gmDate']. values)

There you select the rows and column of df2 what do you want to replace, doing:

df2.loc[(df1['gmDate'].values)==(df2['gmDate'].values),['tf','ta']]

what are these:

    tf  ta
3   0   0
4   0   0
5   0   0
6   0   0
7   0   0
8   0   0
9   0   0

and assign to it:

df1.loc[(df1['gmDate'].values)==(df2['gmDate'].values),['pf','pa']].values

what are these:

array([[ 0,  0],
       [ 0,  0],
       [94, 84],
       [99, 91],
       [ 0,  0],
       [ 0,  0],
       [ 0,  0]])

Doing it also in the opposite case you get the code:

df2.loc[(df1['gmDate'].values)==(df2['gmDate'].values),['tf','ta']]=df1.loc[(df1['gmDate'].values)==(df2['gmDate'].values),['pf','pa']].values
df2.loc[(df1['gmDate'].values)!=(df2['gmDate'].values),['of','oa']]=df1.loc[(df1['gmDate'].values)!=(df2['gmDate'].values),['pf','pa']].values

df2 Output:

    gmDate      t   tw  tf  ta  o   ow  of  oa
0   2012-10-30  WAS 0   0   0   CLE 1   0   0
1   2012-10-30  BOS 0   0   0   MIA 1   0   0
2   2012-10-30  DAL 1   0   0   LAL 0   107 120
3   2012-10-31  DEN 0   0   0   PHI 1   0   0
4   2012-10-31  IND 1   0   0   TOR 0   0   0
5   2012-10-31  HOU 1   94  84  DET 0   0   0
6   2012-10-31  SAC 0   99  91  CHI 1   0   0
7   2012-10-31  SA  1   0   0   NO  0   0   0
8   2012-10-31  DAL 0   0   0   UTA 1   0   0
9   2012-10-31  GS  1   0   0   PHO 0   0   0