Boost Serialization Binary Archive giving incorrect output

Question

I am trying to Serialize a class.

Class definition:

class StartPeerSessionRequest {
public:
    StartPeerSessionRequest();
    virtual ~StartPeerSessionRequest();
    void composeRequestwithHardCodeValues();
    void save();
    stringstream serializedRequest;
    /*boost::serialization::binary_object serlreq;*/

private:
    StartPeerSessionRequest(const StartPeerSessionRequest &);

    uint16_t mProtocolVersion;
    uint16_t mSessionFlags;
    uint16_t mMaxResponseLength;
    string   mMake;
    string   mModel;
    string   mSerialNumber;
    uint8_t  mTrackDelay;
    string   mHeadUnitModel;
    string   mCarModelYear;
    string   mVin;
    uint16_t mVehicleMileage;
    uint8_t  mShoutFormat;
    uint8_t  mNotificationInterval;

    friend class boost::serialization::access;
    template <typename Archive> void serialize(Archive &ar, const unsigned int version);
};

StartPeerSessionRequest::StartPeerSessionRequest() {

    mProtocolVersion      = 1 * 10000 + 14 * 100 + 4;
    mSessionFlags         = 1;
    mMaxResponseLength    = 0;
    mMake                 = "MyMake";
    mModel                = "MyModel";
    mSerialNumber         = "10000";
    mTrackDelay           = 0;
    mHeadUnitModel        = "Headunit";
    mCarModelYear         = "2014";
    mVin                  = "1234567980";
    mVehicleMileage       = 1000;
    mShoutFormat          = 3;
    mNotificationInterval = 1;
}

template <class Archive> void StartPeerSessionRequest::serialize(Archive &ar, const unsigned int version) {
    ar & mProtocolVersion;
    ar & mSessionFlags;
    ar & mMaxResponseLength;
    ar & mMake;
    ar & mModel;
    ar & mSerialNumber;
    ar & mTrackDelay;
    ar & mHeadUnitModel;
    ar & mCarModelYear;
    ar & mVin;
    ar & mVehicleMileage;
    ar & mShoutFormat;
    ar & mNotificationInterval;
}

void StartPeerSessionRequest::save() {
    boost::archive::binary_oarchive oa(serlreq, boost::archive::no_header);
    oa << (*this);
    /*cout<<"\n binary_oarchive :"<<serlreq.size();*/

    boost::archive::text_oarchive ota(serializedRequest, boost::archive::no_header);
    ota << (*this);
    cout << "\n text_oarchive :" << serializedRequest.str() << "size :" << serializedRequest.str().size();
}

serializedRequest.str.size() provides me a length of 87

Actually it should provide me 65 bytes. (I've counted u can figure that out from the constructor)

I suspect it is appending lengths in between.

I have tried using text_archive also it doesnt work.

What I need is to just plain serialize class members as it is.

I guess i need to use some traits or wrappers.

Please let me know

Thanks

Answer 1

Okay, so, just to see how I'd do, I've tried to reach the optimum sizes I calculated on the back of my napkin :

I can see how you'd expect 57, 63, or 75 bytes

 mProtocolVersion = 1*10000+14*100+4; // 2 bytes mSessionFlags = 1; // 2 bytes mMaxResponseLength = 0; // 2 bytes mMake = "MyMake"; // 6 bytes + length mModel = "MyModel"; // 7 bytes + length mSerialNumber = "10000"; // 5 bytes + length mTrackDelay = 0; // 1 byte mHeadUnitModel = "Headunit"; // 8 bytes + length mCarModelYear = "2014"; // 4 bytes + length mVin = "1234567980"; // 10 bytes + length mVehicleMileage = 1000; // 2 byte mShoutFormat = 3; // 1 byte mNotificationInterval = 1; // 1 byte // -------------------------------------- // 51 bytes + 6 x length 

In this instance, I created binary serialization code using Boost Spirit (Karma for serialization and Qi for de-serialization). I made the size of the length field configurable (8,16,32 or 64 bit unsigned).

Here's a working proof of concept: Live On Coliru

generate()

The const generate member function delegates the work to helper functions in a separate namespace:

template <typename Container>
bool generate(Container& bytes) const {
    auto out = std::back_inserter(bytes);

    using my_serialization_helpers::do_generate;
    return do_generate(out, mProtocolVersion)
        && do_generate(out, mSessionFlags)
        && do_generate(out, mMaxResponseLength)
        && do_generate(out, mMake)
        && do_generate(out, mModel)
        && do_generate(out, mSerialNumber)
        && do_generate(out, mTrackDelay)
        && do_generate(out, mHeadUnitModel)
        && do_generate(out, mCarModelYear)
        && do_generate(out, mVin)
        && do_generate(out, mVehicleMileage)
        && do_generate(out, mShoutFormat)
        && do_generate(out, mNotificationInterval);
}

Note that

• do_generate overloads can be freely added as required for future types
• the container can easily be switched from eg std::vector<unsigned char> , to eg boost::interprocess::containers::string<char, char_traits<char>, boost::interprocess::allocator<char, boost::interprocess::managed_shared_memory::segment_manager> > .

parse()

The parse method is very similar except it delegates to do_parse overloads to do the work.

Testing

The test program roundtrips with all possible configurations:

• 8-bit length field, net 57 bytes , with boost serialization: 70
• 16-bit length field, net 63 bytes , with boost serialization: 76
• 32-bit length field, net 75 bytes , with boost serialization: 88
• 64-bit length field, net 99 bytes , with boost serialization: 112

As you can see it's not even that outrageous that the natural Boost Serialization solution would take 107 bytes on my system (it's only 8 bytes more than my last configuration).

Note also, that since the Karma generators all take any output iterator, it should be relatively easy to wire it directly into the low-level Boost Archive operations for performance and to avoid allocating intermediate storage.

Answer 2

You seem to have some highly specific assumptions about how Boost Serialization should serialize to it's proprietary, non-portable binary format.

Boost serialization is much more highlevel, more or less specifically designed to deal with non-POD data. If you insist, you should be able to serialize an array of your POD type directly. In your question, though, the class is not at all POD and hence not bitwise serializable anyway.

For portable archives, see EOS Portable Archive.

Boost Archives have optional flags that suppress the format header:

enum archive_flags {
    no_header = 1,          // suppress archive header info
    no_codecvt = 2,         // suppress alteration of codecvt facet
    no_xml_tag_checking = 4 // suppress checking of xml tags - igored on saving
};

See Archive Models

Here's a backgrounder to see what introduces overhead over simple bitwise serialization:

• Boost C++ Serialization overhead
• how to do performance test using the boost library for a custom library