Can and will C# write objects to the pagefile?

Question

I was wondering if it is possible for C# to write objects to the pagefile.

I already know that a virtual machine for a .NET application is limited to allow one object to only use up 2 GB of ram and will run out of Memory long before that - even on 64-bit version of Windows.

However I need to be able to load a huge amount of strings (not 1 big one but many small ones) and was wondering if it is possible to load them and let them be written to the swap space until they are needed again (unfortunately it is not possible for me to prevent the loading of all the strings because it is forced by an application calling the code I am working on).

To prototype it I tried out to see if the following program will run out of memory as well (which it will), even though it does not try to allocate one huge string:

public static void Main()
{
        ICollection<StringBuilder> builders = new LinkedList<StringBuilder>();
        double used_ram = 2*1024*1024*1024L;
        int blocksize = 12800000;
        int blocks = (int) (used_ram/blocksize);
        for (int i = 1; i < blocks ; i++)
        {
            StringBuilder sb = new StringBuilder(blocksize/2);
            builders.Add(sb);
        }
        Console.ReadLine();
}

I was hoping that the strings would be written to the swap space and was wondering if there is any way I can force the application to do just that.

EDIT: Changed the use of swap file to pagefile (thanks for the clarifications).

Ok so to enhance my question: if the only limitation of the runtime is that ONE object can not allocate more than 2 gb of memory - why is the above code running out of memory - because the list goes over the 2 gb of memory by holding reference to all the string builders?

Answer 1

Short answer: no you can't.

Using the swap is not something applications do: the memory management system of the operating system is responsible of that.

If you need to load more than 2GB of data in your process in one time (and not retrieve data from disk per chunks as necessary), then you have a serious design problem.

---

EDIT : As you're already using a 64-bit OS, make sure you're compiling your application for x64 platforms (or AnyCPU) and your application is not running as a 32-bit process using WOW64.

Answer 2

I think your question amount to:

Can a 32 bit .NET application address more than 2GB of memory.

Yes, it is possible to increase the amount of memory that applications can address by adjusting the split between the memory allocated to user applications and the operating system but I would not recommend it in most cases as it can cause subtle and not so subtle problems with O/S functions, such as networking. For details on the /3GB and /USERVA switches please see this article . This will allow your application to address 3GB (minus overhead) of memory rather than 2GB.

Tools at your disposal are:

• Switching to 64 bit .NET.
• Using memory mapped files.
• Writing your own paging system.
• Using some backing store eg file system, database.

Answer 3

You can utilize all of the windows memory management functions and allocate as much memory as the system will allow, but what you are doing screams for some type of flush to disk system. Even if you could get windows to allocate that large an object you performance would be terrible. There are many out of the box system (called databases) which allow you to put large amounts of data in them and access them it very short order.

Answer 4

You may be doing something wrong. I've written .net applications that use way over 2 GB of memory. If you are running 64 bit windows, there's no reason your application shouldn't be able to use much more memory.

Answer 5

I recommend you should write your own Cache-Class. For an example have a look here

Answer 6

I think what cames closes to your approach on the .Net application level would be a MemoryMappedFile .

But this would mean the application wouldn't try to get all the strings in one go.