Apache POI XSSF的堆空间错误

Question

I am trying to parse a large excel file(.xlsx) using Apache POI XSSF library. After 100,000 rows it throws heap space error. I tried increasing the memory but it does not help. Is there a workaround for this problem? Or can someone suggest me a another library to parse large excel files.

Thanks!

Answer 1

You can use http://poi.apache.org/spreadsheet/how-to.html#xssf_sax_api

Have a look at this thread for details.

Efficient way to search records from an excel file using Apache-POI

Answer 2

Try the latest (stable!) Version from Apache POI.

Alternatives might be smartXLS

Answer 3

When facing the most common OutOfMemoryError, namely the one "java.lang.OutOfMemoryError: Java heap space", some simple aspects must first be understood.

Java applications are allowed to use a limited amount of memory. This limit is specified during application startup. To make things more complex, Java memory is separated different regions named heap space and permgen.

The size of those regions is set during the Java Virtual Machine (JVM) launch by specifying parameters such as -Xmx and -XX:MaxPermSize. If you do not explicitly set the sizes, platform-specific defaults will be used.

So – the “[java.lang.OutOfMemoryError: Java heap space][1]” error will be triggered when you try to add more data into the heap space area, but there is not enough room for it.

Based on this simple description, you have two options

• Give more room to the data structures
• Reduce the size of the data structures used

Giving more room is easy - just increase the heap size by changing the -Xmx parameter, similar to the following example giving your Java process 1G of heap to play with:

java -Xmx1024m com.mycompany.MyClass

Reducing the size of the data structures typically takes more effort, but this might be necessary in order to get rid of the underlying problems - giving more room can sometimes just mask the symptoms and postpone the inevitable. For example, when facing a memory leak you are just postponing the time when all the memory is filled with leaking garbage.

In your case, reading the data in smaller batches and processing each batch at the time might be an option.