What is the theoretical impact of direct index access with “high” memory usage vs. “shifted” index access with “low” memory usage?

Question

Well I am really curious as to what practice is better to keep, I know it (probably?) does not make any performance difference at all (even in performance critical applications?) but I am more curious about the impact on the generated code with optimization in mind (and for the sake of completeness, also "performance", if it makes any difference).

So the problem is as following:

element indexes range from A to B where A > 0 and B > A (eg, A = 1000 and B = 2000).

To store information about each element there are a few possible solutions, two of those which use plain arrays include direct index access and access by manipulating the index:

example 1

//declare the array with less memory, "just" 1000 elements, all elements used
std::array<T, B-A> Foo;
//but make accessing by index slower?
//accessing index N where B > N >= A
Foo[N-A];

example 2

//or declare the array with more memory, 2000 elements, 50% elements not used, not very "efficient" for memory
std::array<T, B> Foo;
//but make accessing by index faster?
//accessing index N where B > N >= A
Foo[N];

I'd personally go for #2 because I really like performance, but I think in reality:

• the compiler will take care of both situations?
• What is the impact on optimizations?
• What about performance?
• does it matter at all?
• Or is this just the next "micro optimization" thing that no human being should worry about?
• Is there some Tradeoff ratio between memory usage : speed which is recommended?

Answer 1

Accessing any array with an index involves adding an index multiplied by element size and adding it to the base-address of the array itself.

Since we are already adding one number to another, making the adjustment for foo[NA] could easily be done by adjusting the base-address down by N * sizeof(T) before adding A * sizeof(T) , rather than actually calculating (AN)*sizeof(T) .

In other words, any decent compiler should comletely hide this subtraction, assuming it is a constant value.

If it's not a constant [say you are using std::vector instread of std::array , then you will indeed subtract A from N at some point in the code. It is still pretty cheap to do this. Most modern processors can do this in one cycle with no latency for the result, so at worst adds a single clock-cycle to the access.

Of course, if the numbers are 1000-2000, probably makes really little difference in the whole scheme of things - either the total time to process that is nearly nothing, or it's a lot becuase you do complicated stuff. But if you were to make it a million elements, offset by half a million, it may make the difference between a simple or complex method of allocating them, or some such.

Also, as Hans Passant implies: Modern OS's with virutal memory handling, memory that isn't actually used doesn't get populated with "real memory". At work I was investigating a strange crash on a board that has 2GB of RAM, and when viewing the memory usage, it showed that this one applciation had allocated 3GB of virtual memory. This board does not have a swap-disk (it's an embedded system). It turns out that some code was simply allocating large chunks of memory that wasn't filled with anything, and it only stopped working when it reached 3GB (32-bit processor, 3+1GB memory split between user/kernel space). So even for LARGE lumps of memory, if you only have half of it, it won't actually take up any RAM, if you do not actually access it.

As ALWAYS when it comes to performance, compilers and such, if it's important, do not trust "the internet" to tell you the answer. Set up a test with the code you actually intend to use, using the actual compiler(s) and processor type(s) that you plan to produce your code with/for, and run benchmarks. Some compiler may well have a misfeature (on processor type XYZ9278) that makes it produce horrible code for a case that most other compilers do this "with no overhead at all".