Ways to store arbitrary data in a std::vector<unsigned char>

Question

I have a third party api that provides the following type (approx.) for attaching arbitrary data to a node in a tree structure:

// 3rd party, cannot be modified
struct AttachedData
{
    std::string key;
    std::vector<unsigned char> data;
};

I think it uses std::vector<unsigned char> because the data could also be written to the disk, but this doesn't matter in my use case as the data only persists at runtime.

There are objects of different types that must be stored in AttachedData (only one object per instance). Since I need the data only at runtime I didn't bother implementing any type of serialization (mostly because of the unnecessary overhead). I decided to use a variant of all possible types and store a pointer to an instance thereof in AttachedData :

using StorageType = std::variant<EventArgs1, EventArgs2, SomeData /*, ...*/>;

AttachedData& attachedData = api.getAttachedData(targetNode, "some key"); // get the correct container
auto storage = new StorageType(someDataInstance); // somehow create the variant on the heap
attachedData.data.resize(sizeof(storage)); // make enough space
std::memcpy(attachedData.data.data(), &storage, sizeof(storage)); // store the pointer to the variant 

On a different place in the code base the data is read and processed:

AttachedData& attachedData = api.getAttachedData(targetNode, "some key"); // get the correct container
StorageType* storage = nullptr;
std::memcpy(&storage, attachedData.data.data(), sizeof(storage)); // read the pointer

// process the stored data
if (auto* args1 = get_if<EventArgs1>(storage))
    // ...
else if (auto* args2 = get_if<EventArgs2>(storage))
    // ...
// ...

// clean up
attachedData.data.clear();
delete storage;

Is there a cleaner or more elegant way to do that? I also thought about using placement new and construct the variant directly in the vector but that would probably cause alignment issues and I don't see any benefits to it either. I think std::variant is a good way to go but I'm open to suggestions in that regard aswell.

Answer 1

UPDATE

The approach proposed in this answer cannot work for OP, since, it turned out, that vectors stored in nodes may be copied, moved, or destructed with no user control. I won't delete the answer for now, since the discussion below clarifies some important details and may be useful for others.

---

ORIGINAL ANSWER

I don't know whether I understood the problem well, but you can create an object of StorageType (ie, std::variant ) in the vector. You just need to resize the vector to fit the object inside:

AttachedData& attachedData = api.getAttachedData(targetNode, "some key");
attachedData.data.resize(sizeof(StorageType));
StorageType* p_obj = new (attachedData.data.data()) StorageType{someDataInstance};

Alignment should be fine as long as all variant's alternatives are not over-aligned .

Then, in the later code, you can obtain access to the stored object as follows:

AttachedData& attachedData = api.getAttachedData(targetNode, "some key"); // get the correct container
StorageType* p_obj = std::launder(reinterpret_cast<StorageType*>(attachedData.data.data()));

---

Note that you need to, somewhere, destruct all the stored objects manually by invoking destructor, or better with std::destroy_at .

---

As for you attempt, it has several problems. For instance, you cannot use memcpy for objects of non-trivially-copyable types, or you are using &storage , which is a pointer to a place in memory where storage pointer is stored (not a pointer where it points to).

UPDATE

According to your additional detail specified in comments, this approach won't work, since you don't control copies, moves, or destruction of vectors. In this case, no approach can generlly work. Once you create an object in the vector, its copy/move/destruction will not correctly involve copy/move/destruction of the object stored inside. Are your variant alternatives trivially-copyable?

Note that std::variant itself is not guaranteed to by trivially-copyable in C++17 even when all its alternatives are. There is a proposal for this problem .