仅在发布时编译时抛出OutOfMemoryException

Question

I have a C# Console application that allocates lots of small objects and arrays. These objects have a short life time and quickly get cleaned up by the garbage collector. To the question "why do you need to allocate so much short life objects, you should avoid this" : the program is used for a heavy AI task and there is no obvious way to get around this for now.

Here is the problem:

If I run the program in debug mode x86 it runs fine and finishes processing everything after a few minutes. In average, it uses 300-400 MB.

If I take that exact same program, but compile and run it in release x86 mode, the memory used by the program quickly reaches 2GB (in a few seconds) so it throws an OutOfMemoryException (which is expected behavior since it's a 32 bit application). Compiling it in release x64 mode does not solve the problem at all, it quickly uses all memory of the computer (8GB), then crashes when a memory allocation fails.

I use SharpDevelop 4.3.3 to build the application. The only differences between debug and release mode are:

• Optimize code (release only)
• Check for arithmetic overflow/underflow (debug only)
• Debug info : full debug information (debug) / no debug information (release)

In all cases no debugger is attached. Program is pretty short and has no compiler directives that would make it run differently when compiled in debug or release. There is no obvious reason to explain the behaviour. When compiling in release mode it looks like garbage collector is never fired (or at least not enough times) and memory is not released.

It seems a similar question has been asked already but it does not seem to be the same issue as mine.

Answer 1

If finally found out the reason. My fault.

I had a one Debug.Assert() method call which not only perform some checking but also did some operation (eg : Debug.Assert(List.Remove()) ).

I assume Debug.Assert() to be executed in both cases (release and debug) and result value to be tested only when in debug mode, but this is wrong. When compiling in release mode, Debug.Assert() calls are totally removed from code.

I put the answer and do not close the question, as it could be useful for someone else.

Answer 2

You could sprinkle calls within your code to force the garbage collector to run. For example:

    GC.Collect();
    GC.WaitForPendingFinalizers();

However, I suspect that the garbage collector is actually running, so this will not have a significant effect. The best practice advice is to NOT try to out-think the garbage collector. I suggest this only to prove that the problem is not there.

Usually when a program consumes all available memory, something is holding onto references to the objects such that the GC cannot free them.

Faced with this problem I would use some sort of memory profiler to figure out what isn't being freed, and what is holding onto it. There are a variety of .NET memory analysis tools out there that you can use. (I'm not sure how good they are on release code). I would probably try using a free trial of JetBrains dotMemory. I have not personally used it, but I have found other analysis tools from them helpful.

By the way, don't apologize for using a design pattern that relies on many short-lived objects. It is a perfectly acceptable design pattern. There are some programming languages for which that is the only way to write code.