[英]A dynamic buffer type in C++?
I'm not exactly a C++ newbie, but I have had little serious dealings with it in the past, so my knowledge of its facilities is rather sketchy.我不是一个 C++ 新手,但我过去很少认真对待它,所以我对它的设施的了解相当粗略。
I'm writing a quick proof-of-concept program in C++ and I need a dynamically sizeable buffer of binary data.我正在用 C++ 编写一个快速的概念验证程序,我需要一个动态大小的二进制数据缓冲区。 That is, I'm going to receive data from a network socket and I don't know how much there will be (although not more than a few MB).
也就是说,我将从网络套接字接收数据,但我不知道会有多少(尽管不会超过几 MB)。 I could write such a buffer myself, but why bother if the standard library probably has something already?
我可以自己编写这样的缓冲区,但是如果标准库可能已经有了一些东西,为什么还要麻烦呢? I'm using VS2008, so some Microsoft-specific extension is just fine by me.
我使用的是 VS2008,所以一些 Microsoft 特定的扩展对我来说很好。 I only need four operations:
我只需要四个操作:
What is the name of the class/function set/whatever that I need?我需要的类/函数集的名称是什么?
Added: Several votes go to std::vector
.补充:几票投给了
std::vector
。 All nice and fine, but I don't want to push several MB of data byte-by-byte.一切都很好,但我不想逐字节推送几 MB 数据。 The socket will give data to me in few-KB large chunks, so I'd like to write them all at once.
套接字将以几 KB 大块的形式向我提供数据,因此我想一次将它们全部写入。 Also, at the end I will need to get the data as a simple char*, because I will need to pass the whole blob along to some Win32 API functions unmodified.
此外,最后我需要将数据作为简单的 char* 获取,因为我需要将整个 blob 传递给一些未修改的 Win32 API 函数。
You want a std::vector
:你想要一个
std::vector
:
std::vector<char> myData;
vector
will automatically allocate and deallocate its memory for you. vector
会自动为你分配和释放它的内存。 Use push_back
to add new data ( vector
will resize for you if required), and the indexing operator []
to retrieve data.使用
push_back
添加新数据(如果需要, vector
将为您调整大小),并使用索引运算符[]
来检索数据。
If at any point you can guess how much memory you'll need, I suggest calling reserve
so that subsequent push_back
's won't have to reallocate as much.如果在任何时候您都可以猜到需要多少内存,我建议调用
reserve
以便后续的push_back
不必重新分配那么多。
If you want to read in a chunk of memory and append it to your buffer, easiest would probably be something like:如果你想读入一块内存并将其附加到你的缓冲区,最简单的可能是这样的:
std::vector<char> myData;
for (;;) {
const int BufferSize = 1024;
char rawBuffer[BufferSize];
const unsigned bytesRead = get_network_data(rawBuffer, sizeof(rawBuffer));
if (bytesRead <= 0) {
break;
}
myData.insert(myData.end(), rawBuffer, rawBuffer + bytesRead);
}
myData
now has all the read data, reading chunk by chunk. myData
现在拥有所有读取数据,逐块读取。 However, we're copying twice.但是,我们复制了两次。
We instead try something like this:我们改为尝试这样的事情:
std::vector<char> myData;
for (;;) {
const int BufferSize = 1024;
const size_t oldSize = myData.size();
myData.resize(myData.size() + BufferSize);
const unsigned bytesRead = get_network_data(&myData[oldSize], BufferSize);
myData.resize(oldSize + bytesRead);
if (bytesRead == 0) {
break;
}
}
Which reads directly into the buffer, at the cost of occasionally over-allocating.它直接读入缓冲区,代价是偶尔会过度分配。
This can be made smarter by eg doubling the vector size for each resize to amortize resizes, as the first solution does implicitly.这可以通过例如将每次调整大小的向量大小加倍以摊销调整大小而变得更智能,因为第一个解决方案是隐式的。 And of course, you can
reserve()
a much larger buffer up front if you have a priori knowledge of the probable size of the final buffer, to minimize resizes.当然,如果您对最终缓冲区的可能大小有先验知识,则可以预先
reserve()
一个更大的缓冲区,以最小化调整大小。
Both are left as an exercise for the reader.两者都留给读者作为练习。 :)
:)
Finally, if you need to treat your data as a raw-array:最后,如果您需要将数据视为原始数组:
some_c_function(myData.data(), myData.size());
std::vector
is guaranteed to be contiguous. std::vector
保证是连续的。
std::vector<unsigned char> buffer;
Every push_back will add new char at the end (reallocating if needed).每个 push_back 都会在最后添加新字符(如果需要重新分配)。 You can call reserve to minimize the number of allocations if you roughly know how much data you expect.
如果您大致知道您期望多少数据,您可以调用reserve 来最小化分配的数量。
buffer.reserve(1000000);
If you have something like this:如果你有这样的事情:
unsigned char buffer[1000];
std::vector<unsigned char> vec(buffer, buffer + 1000);
std::string
would work for this: std::string
将为此工作:
append()
on it with a pointer and a length.append()
来向其附加多字节数据块。data()
on it, and the current length by calling size()
or length()
on it.data()
来获取其内容作为字符数组,并通过对其调用size()
或length()
来获取当前长度。clear()
on it to erase its contents without destroying it.clear()
以擦除其内容而不破坏它。One more vote for std::vector.对 std::vector 再投一票。 Minimal code, skips the extra copy GMan's code do:
最少的代码,跳过额外的副本 GMan 的代码做:
std::vector<char> buffer;
static const size_t MaxBytesPerRecv = 1024;
size_t bytesRead;
do
{
const size_t oldSize = buffer.size();
buffer.resize(oldSize + MaxBytesPerRecv);
bytesRead = receive(&buffer[oldSize], MaxBytesPerRecv); // pseudo, as is the case with winsock recv() functions, they get a buffer and maximum bytes to write to the buffer
myData.resize(oldSize + bytesRead); // shrink the vector, this is practically no-op - it only modifies the internal size, no data is moved/freed
} while (bytesRead > 0);
As for calling WinAPI functions - use &buffer[0] (yeah, it's a little bit clumsy, but that's the way it is) to pass to the char* arguments, buffer.size() as length.至于调用 WinAPI 函数 - 使用 &buffer[0](是的,它有点笨拙,但就是这样)传递给 char* 参数,buffer.size() 作为长度。
And a final note, you can use std::string instead of std::vector, there shouldn't be any difference (except you can write buffer.data() instead of &buffer[0] if you buffer is a string)最后一点,您可以使用 std::string 而不是 std::vector,应该没有任何区别(除非您可以编写 buffer.data() 而不是 &buffer[0] 如果您的缓冲区是字符串)
I'd take a look at Boost basic_streambuf , which is designed for this kind of purpose.我会看看 Boost basic_streambuf ,它是为这种目的而设计的。 If you can't (or don't want to) use Boost, I'd consider
std::basic_streambuf
, which is quite similar, but a little more work to use.如果您不能(或不想)使用 Boost,我会考虑
std::basic_streambuf
,它非常相似,但要使用更多的工作。 Either way, you basically derive from that base class and overload underflow()
to read data from the socket into the buffer.无论哪种方式,您基本上都从该基类派生并重载
underflow()
以将数据从套接字读取到缓冲区中。 You'll normally attach an std::istream
to the buffer, so other code reads from it about the same way as they would user input from the keyboard (or whatever).您通常会将
std::istream
附加到缓冲区,因此其他代码从中读取的方式与用户从键盘(或其他任何方式)输入的方式大致相同。
不是来自 STL 但可能有用的替代方案 - Boost.Circular 缓冲区
使用std::vector ,一个不断增长的数组,保证存储是连续的(你的第三点)。
Regarding your comment "I don't see an append()", ineserting at the end is the same thing.关于您的评论“我没有看到 append()”,最后插入是一回事。
vec.insert(vec.end, vec.insert(vec.end,
If you do use std::vector, you're just using it to manage the raw memory for you.如果您确实使用 std::vector,那么您只是在使用它来为您管理原始内存。 You could just
malloc
the biggest buffer you think you'll need, and keep track of the write offset/total bytes read so far (they're the same thing).您可以只
malloc
您认为需要的最大缓冲区,并跟踪到目前为止读取的写入偏移量/总字节数(它们是相同的)。 If you get to the end ... either realloc
or choose a way to fail.如果你到达终点......无论是
realloc
或选择的方式失败。
I know, it isn't very C++y, but this is a simple problem and the other proposals seem like heavyweight ways to introduce an unnecessary copy.我知道,它不是很 C++y,但这是一个简单的问题,其他建议似乎是引入不必要副本的重量级方法。
The point here is, what you want to use the buffer for.这里的重点是,您想将缓冲区用于什么目的。 If you want to keep structures with pointers the buffer has to be kept fix at the memory address allocated first.
如果要保留带有指针的结构,则必须将缓冲区固定在首先分配的内存地址处。 To circumvent this, you have to use relative pointers and a fixup list for updating the pointers after the final allocation.
为了避免这种情况,您必须使用相对指针和修复列表来在最终分配后更新指针。 This would be worth a class of its own.
这将值得一堂课。 (Didn't find such a thing).
(没有找到这样的东西)。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.