将原始字节数组复制到空字节向量的最有效方法

Question

I have a scenario in which I need to copy the contents of a raw dynamically allocated uint8_t array into a vector (which is guaranteed to be empty whenever this scenario happens).

vector<uint8_t> myVector;
const uint8_t* myRawArray;

It is really important to me that the copy operation is as efficient as possible and portable (various compiler versions might be used).

One approach I thought of using is this:

myVector.reserve(byteCount);
myVector.insert(myVector.begin(), myRawArray, myRawArray + byteCount);

Any ideas on how the speed of that compares to this one:

myVector.resize(byteCount);
memcpy(myVector.data(), myRawArray, byteCount);

I guess memcpy should be fast but then I am forced to use resize which needs to zero-out the memory, so I guess it will slow it down a bit..

Also, any other suggestions?

Answer 1

If you don't need to create the vector before the copy takes place, you could always pass the raw array to the constructor of your vector:

std::vector<uint8_t> myVector(myRawArray, myRawArray + byteCount);

If you do need to construct the vector beforehand, the following is an option:

std::vector<uint8_t> myVector;
// ... do some stuff ...
// Now, we're ready for the copy, and byteCount is known.
myVector.reserve(byteCount);
std::copy(myRawArray, myRawArray + byteCount, std::back_inserter(myVector));

I would suggest using std::copy unless memcpy is proven to be faster. std::copy is safer and more idiomatic in C++ code, but don't be afraid to use memcpy if it really is proven to be faster. The speed difference will most likely change with different compilers.

I hope this helps.

Answer 2

memcpy() is usually written in assembly and it is very optimized so you should know that memcpy will be fast. vector::insert is usually implemented as having a call to memcpy under the hood but it does need to check if there is enough space in the vector for the insertions to take place without any reallocations. I have not profiled this but I bet the first version with the call to reserve is faster.

An alternative to this would be to use std::copy which has been found to be slightly faster than using memcpy in some cases, you can be sure that if possible it also makes a call to memcpy or does something better. So performance issues should not be a problem with it. It will also take care of increasing the size of the vector to match your requirement.

Answer 3

Thanks all for your input to my issue I have resolve the problem by doing the following changes to my structure and implementing it like this

   struct YUV_Buffer
   {
void *pCacheBuf = nullptr;
int frameID = 0;
int height = 0;
int width = 0;
void CopyBuf(BYTE * pBuf, int sizBuf)
{
    pCacheBuf = new BYTE[sizBuf];
    memcpy(pCacheBuf, pBuf, sizBuf);
}

YUV_Buffer(BYTE * pBuf, int nFrameID, int nHeight, int nWidth)
    : frameID(nFrameID), height(nHeight), width(nWidth)
{
    CopyBuf(pBuf, 8 * 1024 * 1024);
}

YUV_Buffer(const YUV_Buffer & yuvbuf)
    :frameID(yuvbuf.frameID), height(yuvbuf.height), width(yuvbuf.width)
{
    CopyBuf((BYTE*)yuvbuf.pCacheBuf, 8 * 1024 * 1024);
}
~YUV_Buffer() {
    delete[]pCacheBuf;
    pCacheBuf = NULL;
}
 };

I then implement it like this:

  YUV_Buffer nBuffer = YUV_Buffer((BYTE*)pSysFrame, pmfxInSurface->Data.FrameOrder, pmfxInSurface->Info.CropH, pmfxInSurface->Info.CropW);
mBuffer.emplace_back(nBuffer);

Hope this may help others also complements to sarabande from expert-exchange for there help and input.

Regards Nigel