reading and writing a vector of structs to file

Question

I've read a few posts on Stack Overflow and a number of other site about writing vectors to files. I've implemented what I feel is working, but I'm having some troubles. One of the data members in the struct is a class string, and when reading the vector back in, that data is lost. Also, after writing the first iteration, additional iterations cause a malloc error. How can I modify the code below to achieve my desired ability to save the vector to a file, then read it back in when the program launches again? Currently, the read is done in the constructor, write in destructor, of a class who's only data member is the vector, but has methods to manipulate that vector.

Here is the gist of my read / write methods. Assuming vector<element> elements ...

Read:

ifstream infile;
infile.open("data.dat", ios::in | ios::binary);
infile.seekg (0, ios::end);
elements.resize(infile.tellg()/sizeof(element));
infile.seekg (0, ios::beg);
infile.read( (char *) &elements[0], elements.capacity()*sizeof(element));
infile.close();

Write:

ofstream outfile;
outfile.open("data.dat", ios::out | ios::binary | ios_base::trunc);
elements.resize(elements.size());
outfile.write( (char *) &elements[0], elements.size() * sizeof(element));
outfile.close();

Struct element:

struct element {
int id;
string test;
int other;        
};

Answer 1

In C++, memory can not generally be directly read and written to disk directly like that. In particular, your struct element contains a string , which is a non- POD data type, and therefore cannot be directly accessed.

A thought experiment might help clarify this. Your code assumes that all your element values are the same size. What would happen if one of the string test values was longer than what you've assumed? How would your code know what size to use when reading and writing to disk?

You will want to read about serialization for more information about how to handle this.

Answer 2

You code assumes all the relevant data exists directly inside the vector, whereas strings are fixed-sized objects that have pointers which can addres their variable sized content on the heap. You're basically saving the pointers and not the text. You should write a some string serialisation code, for example:

bool write_string(std::ostream& os, const std::string& s)
{
    size_t n = s.size();
    return os.write(n, sizeof n) && os.write(s.data(), n);
}

Then you can write serialisation routines for your struct. There are a few design options: - many people like to declare Binary_IStream / Binary_OStream types that can house a std::ostream, but being a distinct type can be used to create a separate set of serialisation routines ala:

operator<<(Binary_OStream& os, const Some_Class&);

Or, you can just abandon the usual streaming notation when dealing with binary serialisation, and use function call notation instead. Obviously, it's nice to let the same code correctly output both binary serialisation and human-readable serialisation, so the operator-based approach is appealing.

If you serialise numbers, you need to decide whether to do so in a binary format or ASCII. With a pure binary format, where portable is required (even between 32-bit and 64-bit compiles on the same OS), you may need to make some effort to encode and use type size metadata (eg int32_t or int64_t?) as well as endianness (eg consider network byte order and ntohl()-family functions). With ASCII you can avoid some of those considerations, but it's variable length and can be slower to write/read. Below, I arbitrarily use ASCII with a '|' terminator for numbers.

bool write_element(std::ostream& os, const element& e)
{
    return (os << e.id << '|') && write_string(os, e.test) && (os << e.other << '|');
}

And then for your vector:

os << elements.size() << '|';
for (std::vector<element>::const_iterator i = elements.begin();
     i != elements.end(); ++i)
    write_element(os, *i);

To read this back:

std::vector<element> elements;
size_t n;
if (is >> n)
    for (int i = 0; i < n; ++i)
    {
        element e;
        if (!read_element(is, e))
            return false; // fail
        elements.push_back(e);
   }

...which needs...

bool read_element(std::istream& is, element& e)
{
    char c;
    return (is >> e.id >> c) && c == '|' &&
           read_string(is, e.test) &&
           (is >> e.other >> c) && c == '|';
}

...and...

bool read_string(std::istream& is, std::string& s)
{
    size_t n;
    char c;
    if ((is >> n >> c) && c == '|')
    {
        s.resize(n);
        return is.read(s.data(), n);
    }
    return false;
}