How to release physical memory when using memory mapped files?

Question

I have this test code whic is simply mapping about 7500 files to virtual memory and then unmapping it. I have done this to check my RAM usage (my virtual memory usage is just fine). When I run this application my physical avaialble memory goes dangerously low and everything really slows down (I am running on a 4 GB RAM machine XP 32 bit SP3). Does any body know how can I solve this ? ( I searched and found that VirtualUnlock may be useful, but when I tried it didn't make any difference). Here is the code:

int COUNT = 7759;

    cout<<"Press a key to start\n";
    getchar();
    int i = 1;
    for(i = 1; i <= COUNT; ++i)
    {
        CString s;
        s.Format(_T("E:\\Images\\CD5\\04221155\\2 (%d)"), i);
        HANDLE hFile = CreateFile(s,
            GENERIC_READ | GENERIC_WRITE,
            FILE_SHARE_READ | FILE_SHARE_WRITE,
            NULL,
            OPEN_EXISTING,
            FILE_ATTRIBUTE_NORMAL,
            NULL);

        if(hFile == INVALID_HANDLE_VALUE)
        {
            TCHAR buf[256] = {0};
            DWORD er = GetLastError();
            FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM, 0, er, 0, buf, 256, 0);
            cout<<"Unable to open file, Error is:";
            wcout<<buf;
            cout<<"\n";
            break;
        }
        DWORD filesize = 0;
        filesize = GetFileSize(hFile,&filesize);

        HANDLE hMapFile = CreateFileMapping(hFile, 
            NULL,
            PAGE_READWRITE,
            0,
            filesize,
            NULL);

        if(hMapFile == NULL)
        {
            cout<<"Unable to create file mapping object. Error is:";
            TCHAR buf[256] = {0};
            DWORD er = GetLastError();
            FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM, 0, er, 0, buf, 256, 0);
            wcout<<buf;
            cout<<"\n";
            break;
        }

        char* pBuf = (char*) MapViewOfFile(hMapFile,
            FILE_MAP_ALL_ACCESS,
            0,
            0,
            filesize);



        if(pBuf == NULL)
        {
            TCHAR buf[256] = {0};
            DWORD er = GetLastError();
            FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM, 0, er, 0, buf, 256, 0);
            cout<<"Unable to map the file to virtual memory, Error is:";
            wcout<<buf;
            CString s1;
            s1.Format(_T("%x"), hMapFile);
            cout<<"Handle is:"<<s1;
            cout<<"\n";
            break;
        }

        if(i % 100 == 0)
        {
            cout<<"Mapped "<<i<<"\n";
        }

        memset(pBuf, 0, filesize);

        //VirtualUnlock(pBuf, filesize);
        UnmapViewOfFile(pBuf);
        CloseHandle(hMapFile);
        CloseHandle(hFile);
    }

    cout<<"finished i is "<<i<<"\n";
    getchar();

Answer 1

You are using MMFs in the absolute worst possible way. Mapping them once, hitting every page and immediately closing the mapping. Your program is generating a massive number of page faults. Something you can easily see in Taskmgr.exe, Processes tab, View + Select Columns to add the page fault columns. Yes, page faults are fairly expensive in getting so many of them is going to noticeably affect the machine's operation. RAM is par for the course, you are using a lot of it.

I realize this is a synthetic test. If your real code looks anything like this (never taking advantage of the caching and the lazy writeback) then don't use MMFs.