most efficient way to convert char vector to string

Question

I have these large pcap files of market tick data. On average they are 20gb each. The files are divided into packets. Packets are divided into a header and messages. Messages are divided into a header and fields. Fields are divided into a field code and field value.

I am reading the file a character at a time. I have a file reader class that reads the characters and passes the characters by const ref to 4 call back functions, on_packet_delimiter, on_header_char, on_message_delimiter, on_message_char. The message object uses a similar function to construct its fields.

Up to here I've noticed little loss of efficiency as compared to just reading the chars and not doing anything with them.

The part of my code, where I'm processing the message header and extracting the instrument symbol of the message, slows down the process considerable.

void message::add_char(const char& c)
{
  if (!message_header_complete) {
    if (is_first_char) {
      is_first_char = false;
      if (is_lower_case(c)) {
        first_prefix = c;
      } else {
        symbol_vector.push_back(c);
      }
    } else if (is_field_delimiter(c)) {
      on_message_header_complete();
      on_field_delimiter(c);
    } else {
      symbol_vector.push_back(c);
    }
  } else {
    // header complete, collect field information
    if (is_field_delimiter(c)) {
      on_field_delimiter(c);
    } else {
      fp->add_char(c);
    }
  }
}

...

void message::on_message_header_complete()
{
  message_header_complete =  true;
  symbol.assign(symbol_vector.begin(),symbol_vector.end());
}

...

In on_message_header_complete() I am feeding the chars to symbol_vector . Once header is complete I convert to string using vector iterator. Is this the most efficient way to do this?

Answer 1

How about:

std::string myStr(myVec.begin(), myVec.end());

Although this works, I don't understand why you need to use vectors in the first place. Just use std::string from the beginning, and use myStr.append() to add characters or strings.

Here's an example:

std::string myStr = "abcd";
myStr.append(1,'e');
myStr.append(std::string("fghi"));
//now myStr is "abcdefghi"

Answer 2

In addition to The Quantum Physicist's answer: std::string should behave quite similar as vector does. Even the 'reserve' function is available in the string class, if you intend to use it for efficiency.

Adding the characters is just as easy as it can get:

std::string s;
char c = 's';
s += c;

You could add the characters directly to your member, and you are fine. But if you want to keep your member clean until the whole string is collected, you still should use a std::string object instead of the vector. You then add the characters to the temporary string and upon completion, you can swap the contents then. No copying, just pointer exchange (and some additional data such as capacity and size...).