Persisting Protobuf messages to Database

Question

What is the right way to persist data defined using protobuf3. I am using golang and Java, both place with support of ORMs. In java with Hibernate and golang with gorm. Both place i need to convert the Generated code to corresponding Entity model. I feel that is more pain full to maintain same object structure in order to be understandable by ORM. Is there any Database which i can use along with protobuf objects as is. Or i can define the relations between objects in the protobuf itself.

Any helps really appreciated.

Answer 1

Your ORM is dealing with objects, by definition. It should not know or care about serialization on the network. I'd suggest deserializing the protobuf message into objects that your ORM is used to and letting it persist them. There's no good reason to couple your persistence tier to the network protocol.

It might make sense to store the protobuf serialization directly if you get rid of JPA and go with a document based solution.

You have to decide how much value JPA is providing for you.

Answer 2

There is a not-straightforward solution to this problem.

Protobuf 3 standardisesJSON mapping for the messages. Once you serialise your message to JSON, you have multiple options for storing it in a database.

The following (and many more) databases can store JSON data:

• MariaDB
• PostgreSQL
• MongoDB

Answer 3

虽然这个问题已经很老了，但事情从那时起就发生了，苹果在 2018 年发布的FoundationDB Record Layer 原生存储了 Protocol Buffer。

Answer 4

In Go, I don't know about gorm, but it seems that with Ent (a competing ORM) Protobufs can be deserialized into exactly the same objects which are used for DB tables/relations. Ent's official tutorial for that.

The caveat is that you specify your Protobuf with Ent's Golang structures, not via the standard proto3 language.