存储大量编译时间常数数据

Question

I hope that this question hasn't been asked before, but I couldn't find any answer after googling for an hour.

I have the following problem: I do numerical mathematics and I have about 40 MB of doubles in the form of certain matrices that are compile time constants. I very frequently use these matrices throughout my program. The creation of these matrices takes a week of computation, so I do not want to compute them on the fly but use precomputed values.

What is the best way of doing this in a portable manner? Right now I have some very large CPP-files, in which I create dynamically allocated matrices with the constants as initialiser lists. Something along the lines:

data.cpp:

namespace // Anonymous
{

// math::matrix uses dynamic memory internally
const math::matrix K { 3.1337e2, 3.1415926e00, /* a couple of hundred-thousand numbers */ };

}

const math::matrix& get_K() { return K; }

data.h

const math::matrix& get_K();

I am not sure if this can cause problems with too much data on the stack, due to the initialiser list. Or if this can crash some compilers. Or if this is the right way to go about things. It does seem to be working though.

I also thought about loading the matrices at program startup from an external file, but that also seems messy to me.

Thank you very much in advance!

Answer 1

I am not sure if this can cause problems with too much data on the stack, due to the initialiser list.

There should not be a problem assuming it has static storage with non-dynamic initialisation. Which should be the case if math::matrix is an aggregate.

Given that the values will be compile time constant, you might consider defining them in a header, so that all translation units can take advantage of them at compile time. How beneficial that would be depends on how the values are used.

I also thought about loading the matrices at program startup from an external file

The benefit of this approach is the added flexibility that you gain because you would not need to recompile the program when you change the data. This is particularly useful if the program is expensive to compile.

Answer 2

A slightly cleaner approach:

// math::matrix uses dynamic memory internally
const math::matrix K { 
   #include "matrix_initial_values.h"
};

And, in the included header,

3.1337e2, 3.1415926e00, 1,2,3,4,5e6, 7.0,...

Considering your comment of "A few hundred thousand" float values: 1M double values takes 8,000,000 bytes, or about 7.6MB. That's not going to blow the stack. Win64 has a max stack size of 1GB, so you'd have to go really, really nuts, and that's assuming that these values are actually placed on the stack, which they should not be given that it's const .

This is probably implementation-specific, but a large block of literals is typically stored as a big chunk of code-space data that's loaded directly into the process' memory space. The identifier ( K ) is really just a label for that data. It doesn't exist on the stack or the heap anymore than code does.