由于64位进程的内存分配过多，Windows冻结

Question

When I run a 64-bit program on Windows (version 7 and up) and the program tries to allocate too much memory (close to 100% of physical memory) the system grinds to a halt. If I don't catch it in time and kill the offending process the system will become unresponsive and a hard reboot is required. Obviously the program shouldn't try to allocate so much memory, but bugs can happen and I want to protect the other running processes from the effects of any such bugs in the code I'm working on when testing/debugging. (This was less of a problem in 32-bits because the logical memory limit provided a safety net from system-wide memory choke.)

So I have two questions:

• Can a process limit its memory consumption? Is there a system call in Windows that can set a memory limit, so that allocations beyond the limit will fail instead of crash the whole system?

• Alternatively, is there any way to set a system-wide per-process memory limit or to set a limit for a particular application?

Here's a sample code that causes my computer to freeze. It was compiled with Visual Studio 10.0 in x86_amd64 configuration and I ran it on a laptop with 16 GB RAM under Windows 8.1. It attempts to allocate 32 GB of memory.

int main(void)
{
    const static int csNumArrays=10, csArraySize=800000000;
    int i, j, **p;

    p=new int*[csNumArrays];
    for (i=0; i<csNumArrays; ++i)
    {
        p[i]=new int[csArraySize];
        for (j=0; j<csArraySize; ++j)
            p[i][j]=j;
    }
    return 0;
}

Answer 1

Probably the best option is for the process to put itself into a Windows job and apply the ProcessMemoryLimit option. (One catch: this won't work if the process is already in a job, eg, because it is being run as a startup script or scheduled task.) You can also use the same technique to limit the memory usage of another process that you launch (or that is already running).

Alternatively, you could try spinning up a thread that uses CreateResourceMemoryNotification to detect a low physical memory condition and terminate the process. But you may find this is subject to false positives. (This should certainly not be left in production code.)

Answer 2

Based on Harry Johnston's helpful answer I put together a bit of code that can be added at the beginning of a program to impose a limit on that program's memory allocation as a factor of the physical memory size.

ULONGLONG aMemSize;
HANDLE aJob=::CreateJobObject(NULL, NULL);
JOBOBJECT_EXTENDED_LIMIT_INFORMATION aLimit;

    // Get physical memory and set process memory limit to 75% of physical memory
GetPhysicallyInstalledSystemMemory(&aMemSize);
aMemSize*=size_t(768);                      // convert kb to bytes and apply 75% factor
memset(&aLimit, 0, sizeof(aLimit));
aLimit.BasicLimitInformation.LimitFlags=JOB_OBJECT_LIMIT_PROCESS_MEMORY;
aLimit.ProcessMemoryLimit=aLimit.PeakProcessMemoryUsed=aLimit.JobMemoryLimit=
                                                aLimit.PeakJobMemoryUsed=aMemSize;
::SetInformationJobObject(aJob, JobObjectExtendedLimitInformation, &aLimit, sizeof(aLimit));
::AssignProcessToJobObject(aJob, ::GetCurrentProcess());