pubsub <-> bigquery with protobuf: bool is getting converted to null or true, not false or true

Question

I have a protobuf pubsub schema being published to bigquery (directly, no dataflow).

in the protobuf, i have a field like:

bool foo = 1;

In the bigquery schema this becomes:

  "name": "foo",
  "type": "BOOLEAN",
  "mode": "NULLABLE"
 },

From my python code, I call publish on the topic w/ a dict (encoded to bytes) that has:

foo: false

this becomes foo: null in the output bigquery table.

if I make if

foo: true

it becomes foo: true in the bigquery table.

this is happening for all of my bool. eg false becomes null, true remains true.

Suggestion on where to look?

Answer 1

This is a known bug with proto3 support that is being actively worked on. You can track progress in the public issue tracker . For now, the workaround is to use proto2 instead of proto3.

Answer 2

TheJSON Mapping section from the Protocol Buffer documentation says:

. . . If a field has the default value in the protocol buffer, it will be omitted in the JSON-encoded data by default to save space.

As false is the default value for protocol Buffer's bools , the above seems to suggest that foo: false by default became foo: null to save space.

I reckon this is a bug because float value 0.0 is also being converted to null in BigQuery .