Not enough memory for JVM heap when calling jni from dynamic library

Question

I have strange situation: we have .dll library (lets call it 'sdk_wrapper'), which is wrapper for java classes, which are opening JRE jvm.dll and use some other .dlls (PKCS11 implementations, to be more specific - but I think it doesn't matter).

Everything works fine when I use sdk_wrapper directly in my c++ program (alias 'run_dll'). But when I pack it into another .dll (to prepare configurations file, initialize library, etc) and export one function which is dong everything the same as program 'run_dll', calling this causes jvm initialization error: Could not reserve enough space for object heap . JVM.dll initialization is doing with -Xmx512m .

Have you any idea what's going on? I readed that JVM needs continous part of memory to initialize, but what's the difference between calling 'sdk_wrapper' directly from program and from another dll? They are at the same place (I mean in the same directory).

Answer 1

As you load libraries into a memory, you end of with memory fragmentation of the address space. Win32 is particularly bad for this.

Note: if you use a 64-bit process this isn't an issue as you will have plenty of virtual memory.