连接和分组填充NaN值

Question

I have this dataframe:

df:
companycode    name    address    A     B     C     ...
1234           asd     qwe,56     Tyh   123   923
1234           asd     qwe,56     Zfhs  4828  01992
6472           yui     iop,56     Retgh 8484  8484
...

I have another one that looks like this:

df2:
companycode    A     B     C       ...
1234           Jid   4123  141
6472           Low   1312  3234
...

name and address are always the same for a single companycode

I want to concatenate or join or merge or append them in a way that in the end it looks like this:

companycode    name    address    A     B     C     ...
1234           asd     qwe,56     Tyh   123   923
1234           asd     qwe,56     Zfhs  4828  01992
6472           yui     iop,56     Retgh 8484  8484
1234           asd     qwe,56     Jid   4123  141
6472           yui     iop,56     Low   1312  3234
...

Since name and address are always the same for a single companycode, basically I want to concat df2 with df in axis=0 and pull to this new rows the name and address from the original df's companycodes. Quite confusing to write but I think that visually it works better.

Any ideas how could I do that?

Answer 1

pd.concat followed by a groupby operation should do it.

df = pd.concat([df1, df2], 0, ignore_index=True)\
                          .groupby('companycode').ffill()
df

       A     B     C address  companycode name
0    Tyh   123   923  qwe,56         1234  asd
1   Zfhs  4828  1992  qwe,56         1234  asd
2  Retgh  8484  8484  iop,56         6472  yui
3    Jid  4123   141  qwe,56         1234  asd
4    Low  1312  3234  iop,56         6472  yui

---

• ignore_index=True is set to create a new index upon concatenation
• Concatenation leaves NaN values in the columns of df2 that didn't previously exist
• Perform a groupby operation on companycode followed by ffill to fill those NaN s with the right values from the same group.

Answer 2

For those with SQL-mindsets, consider a merge with concat (ie, JOIN with UNION ):

mdf = df1[['companycode', 'name', 'address']]\ 
              .merge(df2, on='companycode').drop_duplicates()
finaldf = pd.concat([df1, mdf]).reset_index(drop=True)

print(finaldf)
#    companycode name address      A     B     C
# 0         1234  asd  qwe,56    Tyh   123   923
# 1         1234  asd  qwe,56   Zfhs  4828  1992
# 2         6472  yui  iop,56  Retgh  8484  8484
# 3         1234  asd  qwe,56    Jid  4123   141
# 4         6472  yui  iop,56    Low  1312  3234