Tensorflow：序列对TFRecord的编码和解码是否重要

Question

I have some practice data that I want to encode to a TFRecord format and then decode to tf.features in Tensorflow. My question is very basic, but I could not find a clear answer to this.

Question : Do I need to decode the features in a dataset in the same sequence as they are encoded? In other words, I can't seem to find a way to reference features by field name in a TFRecord. This is really important for 2 reasons.

1. I just wanted to get my assumption validated, so that I know how to avoid breaking my code in the future. Here is some simple code, though this is not a complete example.
2. Python makes a big deal about dictionaries being un-ordered . So how can I guarantee sequence when I am using a data structure that is supposed to be unordered? I was not sure if this was handled in some way that I don't know about.

To encode data into TFRecord format, you can do something like:

#Fields in Dataframe: ['DIVISION','SPORDER','PUMA','REGION']

df = pd.DataFrame(...)
with tf.python_io.TFRecordWriter('myfile.tfrecord') as writer:

    for row in df.itertuples():
        example = tf.train.Example(features=tf.train.Features(feature={
          'feat/division': tf.train.Feature(int64_list=tf.train.Int64List(value=row.DIVISION)),
          'label/sporder': tf.train.Feature(int64_list=tf.train.Int64List(value=row.SPORDER)),
          'feat/puma': tf.train.Feature(bytes_list=tf.train.BytesList(value=[row.PUMA])),
          'feat/region': tf.train.Feature(bytes_list=tf.train.BytesList(value=[row.REGION]))))
        writer.write(example.SerializeToString())

Then to ingest the dataset you would need something like the code below. Notice that the fields are referenced again in order. NOTE: I used the same dictionary keys in the TFRecords versus the decoded form, but I don't think that is necessary--just a convenience. I was not sure if that is the way things have to be? Meaning,

dataset = tf.data.TFRecordDataset('myfile.tfrecord')
dataset = dataset.map(_parse_function)

def _parse_function(example_proto):
    features = {'feat/division': tf.FixedLenFeature((), tf.string, default_value=""),
                'label/sporder': tf.FixedLenFeature((), tf.int64, default_value=0),
                'feat/puma': tf.VarLenFeature(dtype=tf.string),
                'feat/region': tf.VarLenFeature(dtype=tf.string)}

    parsed_example = tf.parse_single_example(example_proto, features)
    parsed_label = parsed_example.pop("label/sporder", None)


    return parsed_example, parsed_label

Answer 1

The tfrecord format uses protobuf for serialization of the struct. You can think about it as a binary json/xml format. Json/xml and protobuf don't care about the order of the fields. So, the order of the feature definitions is not important. It's the same in your snippet because it was just convenient for reading.