从std :: vector <unsigned char>读取二进制数据的最简单方法？

Question

I have a lump of binary data in the form of const std::vector<unsigned char> , and want to be able to extract individual fields from that, such as 4 bytes for an integer, 1 for a boolean, etc. This needs to be, as far as possible, both efficient and simple. eg. It should be able to read the data in place without needing to copy it (eg. into a string or array). And it should be able to read one field at a time, like a parser, since the lump of data does not have a fixed format. I already know how to determine what type of field to read in each case - the problem is getting a usable interface on top of an std::vector for doing this.

However I can't find a simple way to get this data into an easily usable form that gives me useful read functionality. eg. std::basic_istringstream<unsigned char> gives me a reading interface, but it seems like I need to copy the data into a temporary std::basic_string<unsigned char> first, which is not idea for bigger blocks of data.

Maybe there is some way I can use a streambuf in this situation to read the data in place, but it would appear that I'd need to derive my own streambuf class to do that.

It occurs to me that I can probably just use sscanf on the vector's data(), and that would seem to be both more succinct and more efficient than the C++ standard library alternatives. EDIT: Having been reminded that sscanf doesn't do what I wrongly thought it did, I actually don't know a clean way to do this in C or C++. But am I missing something, and if so, what?

Answer 1

You have access to the data in a vector through its operator[] . A vector's data is guranteed to be stored in a single contiguous array, and [] returns a reference to a member of that array. You may use that reference directly, or through a memcpy.

std::vector<unsigned char> v;
...
byteField = v[12];
memcpy(&intField, &v[13], sizeof intField);
memcpy(charArray, &v[20], lengthOfCharArray); 

EDIT 1: If you want something "more convenient" that that, you could try:

template <class T>
ReadFromVector(T& t, std::size_t offset, 
  const std::vector<unsigned char>& v) {
  memcpy(&t, &v[offset], sizeof(T));
}

Usage would be:

std::vector<unsigned char> v;
...
char c;
int i;
uint64_t ull;
ReadFromVector(c, 17, v);
ReadFromVector(i, 99, v);
ReadFromVector(ull, 43, v);

EDIT 2:

struct Reader {
  const std::vector<unsigned char>& v;
  std::size_t offset;
  Reader(const std::vector<unsigned char>& v) : v(v), offset() {}
  template <class T>
  Reader& operator>>(T&t) {
    memcpy(&t, &v[offset], sizeof t);
    offset += sizeof t;
    return *this;
  }
  void operator+=(int i) { offset += i };
  char *getStringPointer() { return &v[offset]; }
};

Usage:

std::vector<unsigned char> v;
Reader r(v);
int i; uint64_t ull;
r >> i >> ull;
char *companyName = r.getStringPointer();
r += strlen(companyName);

Answer 2

You can use a struct that describes the data you are trying to extract. You can move data from your vector into the struct like this:

struct MyData {
    int intVal;
    bool boolVal;
    char[15] stringVal;
} __attribute__((__packed__));

// assuming all extracted types are prefixed with a one byte indicator.
// Also assumes "vec" is your populated vector
int pos = 0;
while (pos < vec.size()-1) {
    switch(vec[pos++]) {
        case 0: { // handle int
            int intValue; 
            memcpy(&vec[pos], &intValue, sizeof(int));
            pos += sizeof(int); 
            // do something with handled value
            break;
        }
        case 1: { // handle double
            double doubleValue; 
            memcpy(&vec[pos], &doubleValue, sizeof(double));
            pos += sizeof(double); 
            // do something with handled value
            break;
        }
        case 2: { // handle MyData
            struct MyData data; 
            memcpy(&vec[pos], &data, sizeof(struct MyData));
            pos += sizeof(struct MyData); 
            // do something with handled value
            break;
        }
        default: {
            // ERROR: unknown type indicator
            break;
        }
    }
}

Answer 3

If your vector stores binary data, you can't use sscanf or similar, they work on text. For converting a byte for a bool is simple enough

bool b = my_vec[10];

For extracting an unsigned int that's stored in big endian order (assuming your ints are 32 bits):

unsigned int i = my_vec[10] << 24 | my_vec[11] << 16 | my_vec[12] << 8 | my_vec[13];

A 16 bit unsigned short would be similar:

 unsigned short s = my_vec[10] << 8 | my_vec[11];¨

Answer 4

If you can afford the Qt dependency, QByteArray has the fromRawData() named constructor, which wraps existing data buffers in a QByteArray without copying the data. With that byte array, you can the feed a QTextStream .

I'm not aware of any such function in the standard streams library (short of implementing your own streambuf , of course), but I'd love to be proved wrong :)

Answer 5

Use a for loop to iterate over the vector and use bitwise operators to access each bit group. For example, to access the upper four bits of the first usigned char in your vector:

int myInt = vec[0] & 0xF0;

To read the fifth bit from the right, right after the chunk we just read:

bool myBool = vec[0] & 0x08;

The three least significant (lowest) bits can be accesed like so:

int myInt2 = vec[0] & 0x07;

You can then repeat this process (using a for loop) for every element in your vector.