C# Too Much Memory Usage

Question

I have a process going with multiple steps defied. (Let's say generic implementation of a strategy pattern) where all steps are passing a common ProcessParameter object around. (read/write to it)

This ProcessParameter is an object having many arrays and collections. Example:

class ProcessParameter() {
   public List<int> NumbersAllStepsNeed {get; set;}
   public List<int> OhterNumbersAllStepsNeed {get; set;}
   public List<double> SomeOtherData {get; set;}
   public List<string> NeedThisToo {get; set;}
   ...
}

Once the steps finished, I'd like to make sure the memory is freed and not hang around, because this can have a big memory footprint and other processes need to run too.

Do I do that by running:

pParams.NumbersAllStepsNeed = null;
pParams.OhterNumbersAllStepsNeed = null;
pParams.SomeOtherData = null;
...

or should ProcessParameter implement IDosposable, and Dispose method would do that, and then I just need to use pParams.Dispose() (or wrap it in using block)

What is the best and most elegant way to clean the memory footprint of the used data of one process running?

Does having arrays instead of lists change anything? Or Mixed? The actual param type I need is collections/array of custom objects.

Am I looking in the right direction?

UPDATE

Great questions! Thanks for the comments! I used to have this process running as a single run and I could see memory usage go very high and then gradually down to "normal".

The problem came when I started chaining this processes on top of each other with different stating parameters. That is when memory went unreasonably high, so I want to include a cleaning step between two processes and looking for best way to do that.

There is a DB, this params is a sort of "cache" to speed up things.

Good point on IDisposable, I do not keep unmanaged resources in the params object.

Answer 1

Whilst using the Disposal pattern is a good idea, I don't think it will give you any extra benefits in terms of freeing up memory.

Two things that might:

1. Call GC.Collect()

However, I really wouldn't bother (unless perhaps you are getting out of memory exceptions). Calling GC.Collect() explicity may hurt performance and the garbage collector really does do a good job on its own. (But see LOH - below.)

2. Be aware of the Large Object Heap (LOH)

You mentioned that it uses a "big memory footprint". Be aware that any single memory allocation for 85,000 bytes or above comes from the large object heap (LOH). The LOH doesn't get compacted like the small object heap. This can lead to the LOH becoming fragmented and can result in out of memory errors even when you have plenty of available memory.

When might you stray into the LOH? Any memory allocation of 85,000 bytes or more, so on a 64 bit system that would be any array (or list or dictionary) with 10,625 elements or more, image manipulation, large strings etc.

Three strategies to help minimise fragmentation of the LOH:

i. Redesign to avoid it. Not always practical. But a list of lists or dictionary of dictionaries might avoid the limit. This can make the implementation more complex so I wouldn't unless you really need to, but on the plus side this can be very effective.

ii. Use fixed sizes. If all of more of your memory allocations in the LOH are the same size then this will help minimise any fragmentation. For example for dictionaries and lists set the capacity (which sets the size of the internal array) to the largest size you are likely to use. Not so practical if you are doing image manipulation.

iii. Force the garbage collector to compact the LOH:

System.Runtime.GCSettings.LargeObjectHeapCompactionMode = System.Runtime.GCLargeObjectHeapCompactionMode.CompactOnce;   
GC.Collect();

you do need to be using .NET Framework 4.5.1 or later to use that. This is probably the simplest approach. In my own applications I have a couple of instances where I know I will be straying into the LOH and that fragmentation can be an issue and I set

System.Runtime.GCSettings.LargeObjectHeapCompactionMode = System.Runtime.GCLargeObjectHeapCompactionMode.CompactOnce;

as standard in the destructor - but only call GC.Collect() explicitly if I get an out of memory exception when allocating.

Hope this helps.