优化速度：向量队列与向量指针队列

Question

I am trying to store vectors (actually an object that manages the vector) onto a queue so that I can process them later.

Here's my current implementation:

// in constructor:  
q = new boost::lockfree::spsc_queue<MyObject>(num_elements_in_q);
// ...
bool Push(const MyObject& push_me) { return q->push(push_me); }
//  ...  
// in Pop() (i.e., this is how I pop stuff off of the queue)  
MyObject temp;  
q->pop(&temp);  

I am wondering if it would make sense to store pointers instead of the object. Here's how the new code would look like:

// in constructor:  
q = new boost::lockfree::spsc_queue<MyObject*>(num_elements_in_q);
// ...
bool Push(const MyObject& push_me) {
  MyObject* ptr = new MyObject(push_me);
  return q->push(push_me);  
}
//  ...  
// in Pop() (i.e., this is how I pop stuff off of the queue)  
MyObject* ptr;  
q->pop(&ptr);
//  do stuff with ptr
delete ptr;

Which approach is best in terms of minimizing the amount of time that the push operation takes? In general is it best to store the entire MyObject or just have pointers stored (and allocate the memory dynamically)? I realize that by storing the entire MyObject, there's still dynamic memory involved since the vector inside MyObject needs to be resized.

My ultimate goal is to minimize the time pushing takes (as well as any time jitter from one operation to the next), at the expense of memory usage and the time it takes for Pop() to execute (the top version requires a copy in Pop() that is avoided by using pointers).

Thanks for the help. Also, I do not currently have access to a profiler on this system, otherwise I might already have my answer.

Answer 1

Without actually testing it, I would say the memory allocation using new could cost more than copying the whole MyObject. Of course it depends on how MyObject is implemented.

Another thing to consider is that storing object itself may give you some higher cache hit rates, assuming boost::lock_free stores data in a continuous memory. Because all your objects can be read by cpu in a batch and therefore stored in L1 cache together. Using pointer will cause CPU to load things from the memory the pointer point to, and potentially kick other elements in the queue out of cache.

Of course, to be 100% sure you have to measure it.

Answer 2

If speed is the ultimate goal look at using some sort in intrusive pattern. By intrusive, I mean, add linking pointers to each of your objects and use these pointers to construct your queues. The big advantage is that there is zero memory allocation when adding an object to the queue. And if you allocate all your objects in one big block (like using a vector), your objects will remain close together. This means that iterating through the list will be less likely to incur cache misses.

This does mean that you will probably need to implement your own locking on the queue but please bear in mind that properly implemented uncontended mutexes should be more or less as cheap as the atomic operations used for lock free programming.

Take a look at: Boost Intrusive for details of the templated boost implementation.

Answer 3

Given that the only real way to figure out what is going on is to measure, I used a crude way to figure out what my execution times (for both implementations were).

The following are results from a run of 2500 insertions into the queue. Times are in seconds based on a boost::timer surrounding the function call. Note these are average times per call.

For storing whole objects:
Run 1: 0.000343423
Run 2: 0.000338752
Run 3: 0.000339651
Run 4: 0.000320011
Run 5: 0.00034017

For storing pointers:
Run 1: 0.00033717
Run 2: 0.00033645
Run 3: 0.000336106
Run 4: 0.00033674
Run 5: 0.000336841

I then went make and increased the test to 25,000 insertions since I was wondering if there was something going on initially with cache misses and the like. Results are below:

For storing whole objects:
Run 1: 0.00023566
Run 2: 0.000255699
Run 3: 0.000250765
Run 4: 0.000239108
Run 5: 0.000264594

For storing pointers:
Run 1: 0.000317314
Run 2: 0.000316985
Run 3: 0.000414893
Run 4: 0.000334542
Run 5: 0.00033179

So it looks like (and this just my theory) that on the initial Push() calls the vectors found in the objects are properly resized. From there, the copy constructor no longer has to pay the penalty of resizing the vector each time and it becomes a much more efficient process.

Answer 4

Agreed that storing a pointer has to be cheaper than storing something larger than a pointer, in almost every circumstance.

In each case, there appears to be a copy construction of a MyObject. By letting the caller be responsible for the lifetime of the object, there is the opportunity to remove this construction:

1. Offering an rvalue interface would allow the use of move construction instead, which may be considerably more light weight, depending on the representation chosen for MyObject.
2. Alternately, you could pass and queue std::unique_ptr<MyObject> smart pointers instead, forcing the caller to explicitly manage the construction and lifetime guarantees of the objects.