C# Memory Issues

Question

I've got an application that:

• Targets C# 6
• Targets .net 4.5.2
• Is a Windows Forms application
• Builds in AnyCPU Mode beacuse it...
• Utilizes old 32 bit libraries that cannot be upgraded to 64 bit, unmanaged memory
• Uses DevExpress, a third party control vendor
• Processes many gigabytes of data daily to produce reports

After a few hours of use in jobs that have many plots, the application eventually runs out of memory. I've spend quite a long time cleaning up many leaks found in the code and have gotten the project to a state where at the worst case it may be using upwards 400,000K of memory at any given time, according to performance counters. Processing this data has not yielded any issues at this point since data is processed in Jagged arrays, preventing any issues with the Large Object Heap.

Last time this happened the user was using ~305,000K of memory. The application is so "out of memory" that the error dialog cannot even draw the error icon in the MessageBox that comes up, the space where the icon would usually be is all black.

So far I've done the following to clean this up:

• Windows forms utilize the Disposed event to ensure that resources are cleaned up, dispose is called manually when required
• Business objects utilize IDisposable to remove references
• Verified cleanup using ANTS memory profiler and SciTech memory profiler. The low memory usage suggests this is not the case but I wanted to see if I saw anything that could be helpful, I could not
• Utilized the GCSettings.LargeObjectHeapCompactionMode property to remove any fragmentation from processing data that may be fragmented in the Large Object Heap (LoH)

Nearly every article that I've used to get to this point suggests that out of memory actually means out of contiguous address space and given the amount that's in use, I agree with this. I'm not sure what to do at this point since from what I understand (and am probably very wrong about) is that the garbage collector clears this up to make room as the process moves along, with the exception of the LoH, which is cleaned up manually now using the new LargeObejctHeapCompactionMode property introduced in .net 4.5.1.

What am I missing here? I cannot build to 64 bit due to the old 32 bit libraries that contain proprietary algorithms that we do not have access to even dream of producing a 64 bit version of. Are there any modes in these profiles I should be using to identify exactly what is growing out of control here?

If this address space cannot be cleared up does this mean that all c# applications will eventually run "out of memory" because of this?

Answer 1

Nearly every article that I've used to get to this point suggests that out of memory actually means out of contiguous address space and given the amount that's in use, I agree with this.

This is a reasonable hypothesis, but even reasonable hypotheses can be wrong. Yours probably is wrong . What should you do?

Test it with science . That is, look for evidence that falsifies your hypothesis. You want to assume that it is anything else , and be forced by the evidence you've gathered that your hypothesis is not false.

So:

• at the point where your application runs out of memory, is it actually out of contiguous free pages of the necessary size? It sure sounds like your observations do not indicate that this is true, so the hypothesis is probably false.

What is other evidence that the hypothesis might be false?

• "After a few hours of use in jobs that have many plots , the application eventually runs out of memory."
• "Uses DevExpress, a third party control vendor"
• "the error dialog cannot even draw the error icon in the MessageBox"

None of this sounds like an out of memory problem. This sounds like a third party control library leaking OS handles for graphics objects. Unfortunately, such leaks usually surface as "out of memory" errors and not "out of handles" errors.

So, that's a new hypothesis. Look for evidence for and against this hypothesis too. You're doing a good job by using a memory profiler. Use a handle profiler next .

If this address space cannot be cleared up does this mean that all c# applications will eventually run "out of memory" because of this?

Nope. The GC does a good job of cleaning up managed memory; lots of applications have no problem running forever without leaking.