Do oneof fields show up on the wire? Is it safe to move a field _out of_ a oneof

Question

https://developers.google.com/protocol-buffers/docs/proto3#oneof doesn't explicitly say that oneof s don't show up on the wire (and it's members are like optional fields), but it heavily hints at it.

If this is true, then it seems it would always be a safe backwards-compatible to move a field out of a oneof without anything getting cleared (unlike moving into a oneof), but the documentation says:

Move fields into or out of a oneof: You may lose some of your information (some fields will be cleared) after the message is serialized and parsed. However, you can safely move a single field into a new oneof and may be able to move multiple fields if it is known that only one is ever set.

It doesn't distinguish between the two. Is this just an oversight?

It also doesn't mention whether it's safe to "inline" the fields inside of a oneof and remove the oneof entirely.

Also, if a oneof contains multiple primitive fields, all of which are default value, how does it know which one is set?

Is there somewhere I can find more details about safe backwards-compatible oneof changes? The official documentation is surprisingly lacking

Answer 1

Personally speaking, I find that whole Backward Compatibility section to be as clear as mud. I can't decide if, when it talks about Tag Re-Use Issues , whether moving / deleting is referring to editing the .proto , or setting / clearing fields in an instantiated object in one's program.

The first part (the para directly under the title Backward Compatibility ) is OK. If a .proto is edited, removing a field from the oneof , the new program reading old data might encounter that field in that old data. There's nothing it can do, except ignore the field and the resultant object will say not set .

A tag is the combination of a field number and a wire type (see this part of the docs ). So if one edited the contents of a oneof in one's .proto file, and reused a field number for a totally different field type, things could get confusing when deserialising old data. The old data could contain tags comprising of the same field number but different wire type .

Moving Fields

As for moving fields into / out of a oneof ; it makes sense in the context of an instantiated object. Suppose that object has been created as a result of deserialising wire data from a file / stream / whatever. That object will, most likely, have one of the fields set. If the program then goes on to set another field in that object, the original field is going to get cleared (which may well be a delete ). However, with a shiny new freshly allocated object, it contains no set fields by default. So, moving a field into it won't delete / null any other field.

My interpretation of "and may be able to move multiple fields if it is known that only one is ever set" is that objects representing fields can be in a "not set" state. Thus moving a collection of fields into a single oneof object works provided that all except one are in this "not set" state. Perhaps the "may" is depenedent on whether this is compiled to C++ or Java or C# or whatever, perhaps there's language differences in how fields are represented and whether or not they can have a "not set" state. For example, C# supports nulls (eg you can have a nullable int), but C++ doesn't.

Though, what this has to do with backward compatibility is not clear to me. For it to have anything to do with backward compatibility, surely we're talking about editing a .proto file and having a new progam deal with data created with an older pre-edit version of the program? That would then go back to a simple matter of whether or not a field number has or has not been reused at any point in the history of the .proto file.

Deleting Fields

I don't know what they're getting at in "Delete a oneof field and add it back". It would be very helpful if they could show the order of the steps they're thinking of.

Sorry to not be very helpful. Oneofs do not show up on the wire as is evident here , so all that is going appear on the wire is the single field that is set.

Also, if a oneof contains multiple primitive fields, all of which are default value, how does it know which one is set?

See this part of the docs and especially this . Note the accessor methods, which will ensure that the object's internal state as to which field is set gets updated. Other langauges that support nullable primitives might use that too.