Python Openpyxl out of memory during read write operation

Question

I have a large xl file(45 MB: 15 sheets, each with exactly 7 columns and 65536 rows). I am trying to append all the data from these 15 sheets into 1 sheet on a separate workbook. I'm using read_only=True for the base xl sheet and write_only=True for the Workbook to be written to.

The issue I am facing is that the after the code is done processing with 11 sheets(check code below), it throws an out of memory error:

Done with sheet[ 9 ] Done with sheet[ 10 ] MemoryError

Using the code below:

import openpyxl as xl
wb1 = xl.load_workbook(path,read_only=True)
wb2 = xl.Workbook(write_only=True)
wb2_ws1=wb2.create_sheet()
sheets = wb1.sheetnames

print("Combine function read only")
for j in range(len(sheets)):

    for row in wb1[sheets[j]].iter_rows(min_row=2 if j != 0 else 1,max_row=wb1[sheets[j]].max_row, min_col=None, max_col=None,values_only=False):
        wb2_ws1.append(cell.value for cell in row)
    print("Done with sheet[",j,"]")

wb2.save(dest_filename)

Any suggestions on how to make this work?

Answer 1

Do you have lxml installed? – Charlie Clark 57 mins ago

@CharlieClark - installing lxml worked. – Vishwesh Chaubal just now Edit