Python Pandas Append to existing excel with matched columns

Question

I am trying to append a dataframe to an existing excel file. Below code checks if the file exists and appends it. But the problem is, if there is one less column in my file, it cannot match the content to the right column. How can I make pandas check the existing column names and match it with my dataframe before appending it?

    if os.path.isfile(output_file):
    book = load_workbook(output_file)
    writer = pd.ExcelWriter(output_file, engine='openpyxl', mode='a')
    writer.book = book
    writer.sheets = {ws.title: ws for ws in book.worksheets}

    for sheetname in writer.sheets:
        source_df.to_excel(writer,
            sheet_name=sheetname, 
            startrow=writer.sheets[sheetname].max_row, 
            columns= ['A','B','D'],
            index = False,
            header= False)

    writer.save()
else:
    source_df.to_excel(output_file, index=False)

Existing Excel:

A B C D
1 2 3 4
5 6 7 8

My Data Frame:

A B D
1 2 3

What I want:

A B C D
1 2 3 4
5 6 7 8
1 2   3

What I get (It writes Column D to the Column C)

A B C D
1 2 3 4
5 6 7 8
1 2 3

Answer 1

A possible alternative to your approach is to do the following:

Suppose you dataframe looks like this:

A  B  D
0  1  2  3
1  1  2  3
2  1  2  3

and that you excel-file (call it filetoupdate) looks like this:

Then the following code will produce:

df_append=pd.read_csv(r"C:\users\k_sego\df2.csv", sep=";" )

to_update = {"shee": df_append}  #I called the sheet 'shee'

file_name = r"C:\....\filetoupdate.xlsx"
excel_reader = pd.ExcelFile(file_name)
excel_writer = pd.ExcelWriter(file_name)

for sheet in excel_reader.sheet_names:
    sheet_df = excel_reader.parse(sheet)
    append_df = to_update.get(sheet)

    if append_df is not None:
        sheet_df = pd.concat([sheet_df, append_df], axis=0)

    sheet_df.to_excel(excel_writer, sheet, index=False)

excel_writer.save()

this excel-file