Why is my application working fine in local and failing in server with java.lang.OutOfMemoryError: nativeGetNewTLA for the same amount of data?

Question

This application that I'm working on has a feature to download the reports(XLSX), DynamicReports 4.0 is the API we are using for it. The application crashes with the following error:

weblogic.kernel.Default failed to schedule a request due to 
java.lang.OutOfMemoryError: nativeGetNewTLA
at weblogic.work.SelfTuningWorkManagerImpl.getWorkAdapter(SelfTuningWorkManagerImpl.java:252)
    at weblogic.work.SelfTuningWorkManagerImpl.schedule(SelfTuningWorkManagerImpl.java:148)
    at weblogic.timers.internal.TimerManagerFactoryImpl$WorkManagerExecutor.execute(TimerManagerFactoryImpl.java:133)
    at weblogic.timers.internal.TimerManagerImpl.execute(TimerManagerImpl.java:621)
    at weblogic.timers.internal.TimerThread$Thread.run(TimerThread.java:277)

when we try to download the reports.

The exact point where it crashes is when it tries to build the report(it has three sheets within it). This happens only with larger volume of data(more than 4MB), for lesser volume it just works fine.

We are using the following server/jvm

Server - Weblogic 10.3.6
JVM- Jrockit 1.6

It's strange that this application works fine in development environment(Windows XP, 64 bit, 3GB Ram) with the default memory args(even for a report with ^MB of data) where as it crashes in the server, which is windows 2008 R2 with 4GB Ram. JVM being used is the same as development env. We tried tuning the memory by setting the following args:

-XXtlaSize:min=16k,preferred=32k -XXlargeObjectLimit:32k

-XX:PermSize=48m

-XX:MaxPermSize=128m

Tried increasing and decreasing the TLA size only to fail.

Why is it working well in development and failing in server? how do I figure out the root cause and solution?

Note: We cannot use profilers on server, however we tried JrockitMission Control that is available with weblogic package and it was of no much help.

Answer 1

Without knowing too much about your program OR your systems (dev and prod) this is just a hypothesis, but in case it is wrong you might still find useful information here that can help you.

So you have two systems:

• DEV: Windows XP, 64 bit, 3GB Ram)

• PROD: which is windows 2008 R2 with 4GB Ram

The error you experience is: OutOfMemoryError: nativeGetNewTLA

Even though you tried tuning TLA settings, it did not help.

I think your problem is not the Thread-Local Area itself. TLA is on heap. Not being able to allocate TLA to a new thread simply means that your app run out of heap memory. So we should focus on the heap memory usage. Since you only specified the physical RAM and the op system I assume that you are using default memory settings. In this case your DEV setup is better off than your PROD. Why? Because your DEV env is 64 bit. If you check the JRockit documentation on default values , you will see that:

Windows on a 64 bit platform - 75% of total physical memory up to 2 GB

Windows on a 32 bit platform - 75% of total physical memory up to 1 GB

Likely your application would actually need a heapsize over 1Gb, and it does not get it on the server. To fix that, you need to manually allocate more max memory available for heap by using:

 -Xmx:2g