Large Excel File - Upload and Import - Java

Question

all

I want to upload and import a large excel files having more than one million records in to my Java program.

I can easily import small files using Apache POI in to my system , but when i starts with large files application throws and out of memory error,

i searched google and found many threads on so , i tried everything but could not get around of this.

can anybody give me solution for my particular problem, import time is not an issue for me, right now also i can bear with performance issue as well,

just want to import this data in to my existing system without oem error.

I have very good configuration on my system and java has enough memory to use so hardware is not an issue.

Thanks

Answer 1

You'll want to stream the data so that you don't need to store all the records in memory at once. POI does support streaming (see XSSF and SAX event API ). As you read the data, ship it off to wherever you need to (database or wherever, you did not specify) -- with the streaming API you should not read all the data into memory before processing it.

You could also export the data to a CSV file, and then use a regular FileInputStream to read the file and process each record as it is read.