Copy columns from one excel file to another excel file sheet using python

Question

I have two excel sheets, I am trying to copy contents from one excel sheet to another using pandas.

First_excel sheet -

Name    Class   City    Date    Misc    Misc2   Blank Col
                 AA             xxx      12      --
                 AA             xx       32      --
                 BB             yyyyy    54      --
                 BB             zz       23      -- 
                 CC             yy       54      --
                 CC             ww       32      --

Second_excel sheet -

Name    Class   Date    City
Tom     Q,W   01-11-20   AA
Jerry   W     05-03-19   AA
Don     E,R   06-05-20   BB
Rob     T,Y   02-01-20   BB
Mike    W     05-03-18   CC
Ann     O,p   04-09-20   CC

Final Expected Sheet -

Name    Class   City    Date    Misc    Misc2   Blank Col
Tom     Q,W     AA   01-11-20   xxx     12       --
Jerry   W       AA   05-03-19   xx      32       --
Don     E,R     BB   06-05-20   yyyyy   54       --
Rob     T,Y     BB   02-01-20   zz      23       --
Mike    W       CC   05-03-18   yy      54       --
Ann     O,p     CC   04-09-20   ww      32       --



df1 = pd.read_excel("new_excel_file.xlsx", sheet_name = "Sheet1")
df2 = pd.read_excel("new_excel_file.xlsx", sheet_name = "Sheet2")
result = pd.concat([df1,df2])

This is the resultant dataframe I got

    Name    Class   City    Date        Misc    Misc2   Blank Col   
0   NaN     NaN     AA      NaT         xxx     12.0    --          
1   NaN     NaN     AA      NaT         xx      32.0    --          
2   NaN     NaN     BB      NaT         yyyyy   54.0    --          
3   NaN     NaN     BB      NaT         zz      23.0    --          
4   NaN     NaN     CC      NaT         yy      54.0    --          
5   NaN     NaN     CC      NaT         ww      32.0    --          
0   Tom     Q,W     AA      2020-11-01  NaN     NaN     NaN         
1   Jerry   W       AA      2019-03-05  NaN     NaN     NaN         
2   Don     E,R     BB      2020-05-06  NaN     NaN     NaN         
3   Rob     T,Y     BB      2020-01-02  NaN     NaN     NaN         
4   Mike    W       CC      2018-03-05  NaN     NaN     NaN         
5   Ann     O,p     CC      2020-09-04  NaN     NaN     NaN         

My ideas was to replace NaN with actual values from df2 or second_excel in their positions. Please help me to get my expected output.

Answer 1

You can use fillna to merge the two columns with same name then append the last column. First load the files:

df1 = pd.read_excel("new_excel_file.xlsx", sheet_name = "Sheet1")
df2 = pd.read_excel("new_excel_file.xlsx", sheet_name = "Sheet2")

Fill empty columns in df1 with df2:

fill = df1.fillna(df2[['Name', 'Class', 'Date']])

Then join the last column:

result = teste.join(df2[['City']])

Edit

Since you edited your post, just use fill = df1.fillna(df2) and it will do

The output:

    Name Class  City                 Date   Misc  Misc2 Blank Col 
0    Tom   Q,W    AA  2020-11-01 00:00:00    xxx     12        -- 
1  Jerry     W    AA  2019-03-05 00:00:00     xx     32        -- 
2    Don   E,R    BB  2020-05-06 00:00:00  yyyyy     54        --  
3    Rob   T,Y    BB  2020-01-02 00:00:00     zz     23        --  
4   Mike     W    CC  2018-03-05 00:00:00     yy     54        --  
5    Ann   O,p    CC  2020-09-04 00:00:00     ww     32        --