Rule of thumb for when passing by value is faster than passing by const reference?

Question

Suppose I have a function that takes an argument of type T . It does not mutate it, so I have the choice of passing it by const reference const T& or by value T :

void foo(T t){ ... }
void foo(const T& t){ ... }

Is there a rule of thumb of how big T should become before passing by const reference becomes cheaper than passing by value? Eg, suppose I know that sizeof(T) == 24 . Should I use const reference or value?

I assume that the copy constructor of T is trivial. Otherwise, the answer to the question depends on the complexity of the copy constructor, of course.

I have already looked for similar questions and stumbled upon this one:

template pass by value or const reference or...?

However, the accepted answer ( https://stackoverflow.com/a/4876937/1408611 ) does not state any details,it merely states:

If you expect T always to be a numeric type or a type that is very cheap to copy, then you can take the argument by value.

So it does not solve my question but rather rephrases it: How small must a type be to be "very cheap to copy"?

Answer 1

If you have reason to suspect there is a worthwhile performance gain to be had, cut it out with the rules of thumb and measure . The purpose of the advise you quote is that you don't copy great amounts of data for no reason, but don't jeopardize optimizations by making everything a reference either. If something is on the edge between "clearly cheap to copy" and "clearly expensive to copy", then you can afford either option. If you must have the decision taken away from you, flip a coin.

A type is cheap to copy if it has no funky copy constructor and its sizeof is small. There is no hard number for "small" that's optimal, not even on a per-platform basis since it depends very much on the calling code and the function itself. Just go by your gut feeling. One, two, three words are small. Ten, who knows. A 4x4 matrix is not small.

Answer 2

Passing a value instead of a const reference has the advantage that the compiler knows the value isn't going to change. "const int& x" doesn't mean the value cannot change; it only means that your code is not allowed to change it by using the identifier x (without some cast that the compiler would notice). Take this awful but perfectly legal example:

static int someValue;

void g (int i)
{
    --someValue;
}

void f (const int& x)
{
    for (int i = 0; i < x; ++i)
        g (i);
}

int main (void)
{
    someValue = 100;
    f (someValue);
    return 0;
}

Inside function f, x isn't actually constant! It changes every time that g (i) is called, so the loop only runs from 0 to 49! And since the compiler generally doesn't know whether you wrote awful code like this, it must assume that x might change when g is called. As a result, you can expect the code to be slower than if you had used "int x".

The same is obviously true for many objects as well that might be passed by reference. For example, if you pass an object by const&, and the object has a member that is int or unsigned int, then any assignment using a char*, int*, or unsigned int* might change that member, unless the compiler can prove otherwise. Passed by value, the proof is much easier for the compiler.

Answer 3

The most appropriate rule of thumb in my opinion is pass by reference when :

sizeof(T) >= sizeof(T*)

The idea behind this is that when you take by reference, at worst your compiler might implement this using a pointer.

This of course doesn't take into account the complexity of your copy constructor and move semantics and all the hell that can be created around your object life cycle.

Also if you don't care about micro optimisations you can pass everything by const reference, on most machines pointer are 4 or 8 bytes, very few types are smaller than that and even in that case you would lose a few (less than 8) bytes copy operation and some indirections which in modern world is most likely not gonna be your bottleneck :)

Answer 4

I believe I would choose to pass by value whenever possible (that is: when the semantics dictate that I do not need the actual object to work on). I would trust the compiler to perform the appropriate moves and copy-elision.

After my code is semantically correct, I would profile it to see if I am making any unnecessary copies; I would modify those accordingly.

I believe that this approach would help me focus on the most important part of my software: correctness. And I would not get on the way of the compiler---interfere; inhibit---to perform optimizations (I know I cannot beat it).

Having said that, nominally references are implemented as pointers. So in a vaccum, without considering semantics, copy-elisions, move semantics, and stuff like that, it would be more "efficient" to pass by pointer/reference anything whose size is larger than the pointer.

Answer 5

For an abstract "T" in an abstract "C++" the rule of thumb would be to use the way that better reflects the intention, which for an argument that isn't modified is almost always "pass by value". Besides, concrete real world compilers expect such an abstract description and gonna pass your T in the most efficient way, regardless of how you do this in the source.

Or, to talk about naivie compilation and composition , "very cheap to copy" is "anything you can load in a single register". Doesn't get any cheaper than that really.

Answer 6

If you're going to use a "rule of thumb" for by-value vs. by-const-reference, then do this:

• pick ONE approach and use it everywhere
• agree upon which one among all your coworkers
• only later, in the "hand-tuning performance" phase, start changing things
• and then, only change them if you see a measurable improvement

Answer 7

The guys here are correct that most of the time it doesn't matter when the sizeof the struct/class is small and there's no fancy copy constructor. However that's no excuse for being ignorant. Here's what happens in some modern ABIs like x64. You can see that on that platform, a good threshold for your rule of thumb is to pass by value where the type is a POD type and sizeof() <= 16, since it will get passed in two registers. Rules of thumb are good, they keep you from wasting time and limited brainpower on the little decisions like this that don't move the needle.

However, sometimes it will matter. When you've determined with a profiler that you have one of those cases (and NOT before unless it's super obvious!), then you need to understand the low level details - not hear comforting platitudes about how it doesn't matter, which SO is full of. Some things to keep in mind:

• Passing by value tells a compiler that the object doesn't change. There are evil kinds of code, involving threads or globals where the thing pointed to by the const reference is being changed somewhere else. Although surely you don't write evil code, the compiler may have a tough time proving that, and have to generate very conservative code as a result.
• If there are too many arguments to pass by register, then it doesn't matter what the sizeof is, the object will get passed on the stack.
• An object that is small and POD today may grow and gain an expensive copy constructor tomorrow. The person who adds those things probably did not check if it was being passed around by reference or by value, so code that used to perform great may suddenly chug. Passing by const reference is a safer option if you work on a team without comprehensive performance regression tests. You will get bitten by this sooner or later.