std :: vector &lt;std :: vector <unsigned char> &gt;或std :: deque &lt;std :: vector <unsigned char> &gt;？

Question

I have an existing algorithm and I need to optimize it sligtly if it is possible. Changing a lot in this algorithm is not an option at the moment. The algoritm works with instance of std::vector< std::vector<unsigned char> > . It looks like this:

typedef std::vector<unsigned char> internal_vector_t;
std::vector< internal_vector_t > internal_vectors; 

while (fetching lots of records) {
   internal_vector_t tmp;
   // reads 1Mb of chars in tmp...
   internal_vectors.push_back(tmp);
   // some more work
}

// use this internal_vectors

The algorithm inserts a lot of times in internal_vectors instances of internal_vector_t using push_back(). Most of instances of internal_vector_t have size 1 Mb . Since the size of the internal_vectors is unknown no reserve() is done beforehand.

The first thing that I don't understand is what is happening when internal_vectors reachs its current capacity, needs to allocate a new block and copy its current content in the bigger block of memory. Since most of the blocks are 1Mb in size copying is a long operation. Should I expect that a compiler (gcc 4.3, MS VC++ 2008) will manage to optimize it in order to avoid copying ?

If copying is unavoidable will changing to std::deque help ? I consider std::deque because I still need accessing by index like internal_vectors[10]. Like this:

typedef std::vector<unsigned char> internal_vector_t;
std::deque< internal_vector_t > internal_vectors; 
// the same while

As far as I understand std::deque does not need relocate that was once allocated. Am I right that std::deque in this situation will requere less allocation and copying on push_backs?

---

Update:
1) According to DeadMG MSVC9 does this type of optimization (The Swaptimization - TR1 Fixes In VC9 SP1 ). gcc 4.3 probably doesn't do this type of optimization.

2) I have profiled the version of the algorithm that use std::deque< std::vector<unsigned char> > and I see that its performace is better.

3) I have also made use of using swap that was suggested by Mark Ransom . Using this improved the performance:

  internal_vector_t tmp; internal_vectors.push_back(empty); tmp.swap(internal_vectors.back()); 

Answer 1

MSVC9 implements something known as "swaptimization" for it's Standard containers. It's a weaker version of move semantics. When the external vector is resized, it will not copy the internal vectors.

However, you'd do best simply upgrading your compiler to MSVC10 or GCC (4.5, I think it is) which will give you move semantics, which makes such operations vastly more efficient. Of course, a std::deque is probably still the smarter container, but move semantics are performance-beneficial in many, many places.

Answer 2

Each time you insert a internal_vector_t into internal_vectors , it is going to make a copy of the internal_vector_t . This will be true whether you use vector or deque . The standard containers always make a copy of the object you're inserting.

You can eliminate the copying by inserting an empty internal_vector_t and then swap the contents of the inserted object with the one you really wanted to insert.

Occasionally the vector will need to resize itself as it runs out of room during an insertion, which would result in objects being copied again. A deque will eliminate this as long as you're always inserting at the beginning or end.

Edit: The advice I gave above can be summarized with these code changes. This code should avoid all copying of the large vectors.

typedef std::vector<unsigned char> internal_vector_t;
std::deque< internal_vector_t > internal_vectors; 
internal_vector_t empty;

while (fetching lots of records) {
   internal_vector_t tmp;
   // reads 1Mb of chars in tmp...
   internal_vectors.push_back(empty);
   tmp.swap(internal_vectors.back());
   // some more work
}

Answer 3

std::deque does not store it's elements contiguously - it breaks it's storage up into a series of constant sized "blocks". This means that when a std::deque runs out of capacity it only needs to allocate a new block of constant size - it does not need to reallocate it's whole internal buffer and move all of it's existing elements.

std::vector on the other hand does maintain contiguous storage, so when it runs out of capacity and reallocates, it does need to move all of it's existing elements - this can be expensive.

std::vector is "smart" about its reallocation scheme, allocating in chunks according to a geometric series (often doubling or increasing the capacity by 1.5 etc). This means that reallocation doesn't occur often.

std::deque may be more efficient in this case since when reallocation does occur it does less work. As always, you'd have to benchmark to get any real numbers.

Your code could probably be improved further in other areas. It seems that at each iteration of the while loop you're creating a new internal_vector_t tmp . It may be more efficient to declare this outside the loop and just ::clear() it's storage at each iteration. You're also copying the whole tmp vector each time you call internal_vectors.push_back(tmp) - you could probably improve on this by just moving the tmp vector via internal_vectors.push_back(std::move(tmp)) - this will just copy a few pointers.

Hope this helps.

Answer 4

Are you indexing the outer vector? If not, how about std::list<std::vector<unsigned char> > ?

Answer 5

A dequeue may be more efficient depending on the implementation. Unlike a vector, a dequeue will not guarantee continuous storage and can thus allocate several separate blocks of memory. Therefore it can allocate more memory without moving elements already added. You should try it and measure the impact.