Creating a Master excel file from dynamic CSV output using Python

Question

I am trying to create a repository "Master" excel file from a CSV which will be generated and overwritten every couple of hours. The code below creates a new excel file and writes the content from "combo1.csv" to "master.xlsx". However, whenever the combo1 file is updated, the code basically overwrites the contents in the "master.xlsx" file. I need to append the contents from "combo1" to "Master" without the headers being inserted every time. Can someone help me with this?

import pandas as pd

writer = pd.ExcelWriter('master.xlsx', engine='xlsxwriter')
df = pd.read_csv('combo1.csv')
df.to_excel(writer, sheet_name='sheetname')
writer.save()

Answer 1

Refer to Append Data at the End of an Excel Sheet section in this medium article: Using Python Pandas with Excel Sheets

(Credit to Nensi Trambadiya for the article)

Basically you'll have to first read the Excel file and find the number of rows before pushing the new data.

reader = pd.read_excel(r'master.xlsx')
df.to_excel(writer,index=False,header=False,startrow=len(reader)+1)

Answer 2

First read the excel file and then need to perform below method to append the rows.

import pandas as pd
from xlsxwriter import load_workbook

df = pd.DataFrame({'Name': ['abc','def','xyz','ysv'],
                   'Age': [08,45,32,26]})
writer = pd.ExcelWriter('master.xlsx', engine='xlsxwriter')

writer.book = load_workbook('Master.xlsx')

writer.sheets = dict((ws.title, ws) for ws in writer.book.worksheets)

reader = pd.read_excel(r'master.xlsx')

df.to_excel(writer,index=False,header=False,startrow=len(reader)+1)

writer.close()

Answer 3

import pandas as pd
from openpyxl import load_workbook

# new dataframe with same columns
df = pd.read_csv('combo.csv')
writer = pd.ExcelWriter('master.xlsx', engine='openpyxl')
# try to open an existing workbook
writer.book = load_workbook('master.xlsx')
# copy existing sheets
writer.sheets = dict((ws.title, ws) for ws in writer.book.worksheets)
# read existing file
reader = pd.read_excel(r'master.xlsx')
# write out the new sheet
df.to_excel(writer, index=False, header=False, startrow=len(reader) + 1)

writer.close()

Note that a Master has to be created before running the script