可调整大小的char缓冲区容器类型，用于C ++

Question

I'm using libcurl (HTTP transfer library) with C++ and trying to download files from remote HTTP servers. As file is downloaded, my callback function is called multiple times (eg every 10 kb) to send me buffer data.

Basically I need something like "string bufer", a data structure to append char buffer to existing string. In C, I allocate ( malloc ) a char* and then as new buffers come, I realloc and then memcpy so that I can easily copy my buffer to resized array.

In C, there are multiple solutions to achieve this.

1. I can keep using malloc , realloc , memcpy but I'm pretty sure that they are not recommended in C++.
2. I can use vector<char> .
3. I can use stringstream .

My use cases is, I'll append a few thousands of items ( char s) at a time, and after it all finishes (download is completed), I will read all of it at once. But I may need options like seek in the future (easy to achieve in array solution (1)) but it is low priority now.

What should I use?

Answer 1

I'd go for stringstream . Just insert into it as you recieve the data, and when you're done you can extract a full std::string from it. I don't see why you'd want to seek into an array? Anyway, if you know the block size, you can calculate where in the string the corresponding block went.

Answer 2

I'm not sure if many will agree with this, but for that use case I would actually use a linked list, with each node containing an arbitrarily large array of char that were allocated using new . My reasoning being:

• Items are added in large chunks at a time, one at a time at the back.
• I assume this could use quite a large amount of space, so you avoid reallocation events when a vector would otherwise need more space.
• Since items are read sequentially, the penalty of link lists being unidirectional doesn't affect you.

Should Seeking through the list become a priority, this wouldn't work though. If it's not a lot of data ultimately, I honestly think a vector would be fine, dispite not being the most efficient structure.

Answer 3

If you just need to append char buffers, you can also simply use std::string and the member function append . On top of that stringstream gives you formatting, functionality, so you can add numbers, padding etc., but from your description you appear not to need that.

Answer 4

I think I'd use a deque<char> . Same interface as vector , and vector would do, but vector needs to copy the whole data each time an append exceeds its existing capacity. Growth is exponential, but you'd still expect about log N reallocations, where N is the number of equal-sized blocks of data you append. Deque doesn't reallocate, so it's the container of choice in cases where a vector would need to reallocate several times.

Assuming the callback is handed a char* buffer and length, the code to copy and append the data is simple enough:

mydeque.insert(mydeque.end(), buf, buf + len);

To get a string at the end, if you want one:

std::string mystring(mydeque.begin(), mydeque.end());

I'm not exactly sure what you mean by seek , but obviously deque can be accessed by index or iterator, same as vector .

Another possibility, though, is that if you expect a content-length at the start of the download, you could use a vector and reserve() enough space for the data before you start, which avoids reallocation. That depends on what HTTP requests you're making, and to what servers, since some HTTP responses will use chunked encoding and won't provide the size up front.

Answer 5

I would use vector<char> . But they will all work even with a seek, so your question is really one of style and there are no definitive answers there.

Answer 6

Create your own Buffer class to abstract away the details of the storage. If I were you I would likely implement the buffer based on std::vector<char> .