C++ memory allocation: 'new' throws bad_alloc?

Question

In my application I save voxel-data (compressed byte array) and trying to load it back again into memory.

But my application crashes in Windows 7 machine (64 bit ,12 GB) giving bad_alloc .

This works fine on Linux ,and even runs on some other Windows 7 machine with 4 GB memory.

Compressed bye size is about 200Mb and uncompressed byte size is about 300Mb.(so only 500 MB reserved before this 'new' statement,which leaves almost 8 GB excluding system memory ). I don't run any other major memory allocation steps before this ,so I don't think memory is fragmented. I use MinGW GCC version 4.5.0

Any tips on this ?

thanks in advance.

QFile fileVol(strVFile);

//Read VolumeData
fileVol.open(QIODevice::ReadOnly);
QDataStream volstream(&fileVol);
QByteArray volCmprsdByteArray;     //This holds the compressed byte array

//Read some additional information
int nx, ny, nz, bbp;
int voltype;
volstream >> nx;
volstream >> ny;
volstream >> nz;
volstream >> bbp;
volstream >> voltype;
volstream >> volCmprsdByteArray;     //read our compressed voxel-data

//we have original uncompressed pixel data
QByteArray volUncmprsdByeArray = qUncompress(volCmprsdByteArray);

int uncompressedSize = volUncmprsdByeArray.size(); // size of the byte array
qDebug("new char for uncompressed data size %d",uncompressedSize);

unsigned char* volumeData=NULL;

//Trying to allocate new memory 
try {
    // ##### breaks here.   ####
    volumeData =new unsigned char[uncompressedSize];   
 }
catch (std::bad_alloc e)
 {
      cout << "lu_solver() bad_alloc: " << e.what() << endl;
      cout << "Size " << uncompressedSize << endl;
      //return;
}

Answer 1

If you are building the program as a 32-bit binary on Windows, you only get 2GB of available address space (the upper half is reserved).

When you consider that the address space can be fragmented by lots of different things being allocated (DLLs getting loaded, thread stacks, dynamic allocation), it's likely that you'll end up having large allocations fail very quickly.

You can try giving the linker the /LARGEADDRESSAWARE flag, which will give you a 3GB address space on correctly configured 32-bit Windows and the full 4GB address space on 64-bit Windows. Really, though, it's a good idea to find a way to reimplement your program so that you can use several smaller allocations instead of one huge one.

Answer 2

For very large allocations you may be running into fragmentation issues: the heap may have, say, 600mb of memory left, but chopped up into 100mb chunks, so that you can't allocate a single 500mb block. Some implementations of new (which really calls malloc() in most cases) are not very smart about failing gracefully when this happens.

With such big allocations, you might have more luck using OS-specific functions like VirtualAlloc() and HeapAlloc() on Windows.

Answer 3

如果您的数据大小是固定的，或具有已知的上限，则可以尝试在堆栈或全局变量中进行静态分配。