Memory Cache Maximum limit in Asp.net

Question

I am developing an asp.net application which needs to search a record from around 5 million records(of around 4GB data). Client is looking for higher performance and decided to memory cache. But I m facing issue while uploading data into memory cache from asp.net. I tried changing application pool settings and made virtual memory as 0, private memory as 0.. Nothing worked out. It is uploading fine till around 1.5GB and throwing out of memory exceptions. There is no issue when I pushed data using console application by unchecking " 32 bit" in build settings in application properties. My Issue with asp.net. I am using .net frame work 4.0 with 4 core server , the memory available in the server is around 49GB. I also tried with enabling 32 bit run on 64 mode in application pool. But nothing changed. Could please suggest me if there is any solution.

Answer 1

As already mentioned by John: Querying 5.000.000 records is the job of a DB and NOT your code. If you configure the DB correctly (let the DB use the memory as a cache, correct indexes, performant SQL-query) I would say with a 99.9% chance the DB will be MUCH faster then anything you can create in ASP.NET.

Anyhow if you REALLY want to do it the other way around, you need to create a 64-bit process.
Checklist for doing that (out of my head - no guarantee for completeness):

• compile (all parts) of the solution as "Any CPU" or "x64"
• run IIS on a 64-bit CPU an OS (which should be the case with 49 GB RAM available)
• Set the Application-Pool to run as a 64-bit process with no memory limit:
  • Application Pools -> "Your Pool" -> Advanced Settings...
    -> Enable 32-bit Application -> False
    -> Private Memory Limit (KB) -> 0
    -> Virtual Memory Limit (KB) -> 0

Answer 2

Thank you guys. I agree that i can do it in my DB. I am handling a huge volume of requests per sec. My DB query is like : select a,b,c,d,e from table1 where id = primary key... It is very simple and efficient query.. although this is efficient.. it is not giving the required performance. So we decided to use cache. Now we resolved the issue by creating a windows service ( which creates a proxy and hosts cache) and web application separately. web application internally call this windows service.. It is working now. Thank you for all suggestions.