how to parse a protobuf3 message without knowing its type at compile time with golang?

Question

here's a scenario:

you're implementing in golang a generic component that can may be used with any type of proto message (binary serialization) and need to deserialize binary proto data without knowing its type at compile time.

for instance, i encountered this issue while writing a generic kafka json archiver, the component would:

• receive a message type (string) from configuration and a name of a kafka topic
• would need to create the binary -> memory deserializer and the memory -> json serializer at runtime.

how do you get a deserializer for your binary bytes just from the message name?

Answer 1

golang proto libraries have a helper utility for this purpose:

// MessageType returns the message type (pointer to struct) for a named message.
// The type is not guaranteed to implement proto.Message if the name refers to a
// map entry.
func MessageType(name string) reflect.Type {
   // ....
}

to use it you can use a method similar to this:

func getProto(messageType string, messageBytes []byte) proto.Message {
    pbtype := proto.MessageType(messageType)
    msg := reflect.New(pbtype.Elem()).Interface().(proto.Message)
    proto.Unmarshal(messageBytes, msg)
    return msg
}

i have put a full example of this on github: https://github.com/rotemtam/pbreflect-example