Why does reading a simple tfrecord crash python?

Question

I wrote the following snippet for writing and reading a TFRecord. The last tf.run() statement stops python from responding to anything. What is the reason for this?

fn = 'tmp.tfrecord'
seqs = [[1,2,3], [0,1,0]]

writer = tf.python_io.TFRecordWriter(fn)

for seq in seqs:
    ex = tf.train.Example(features=
        tf.train.Features(feature={'seq': tf.train.Feature(int64_list=tf.train.Int64List(value=seq))}))
    writer.write(ex.SerializeToString())

writer.close()


# Now read the written records:

filename_queue = tf.train.string_input_producer([fn])

reader = tf.TFRecordReader()
key, serialized_example = reader.read(filename_queue)

features = { 'seq': tf.FixedLenFeature([], dtype=tf.int64) }

ex_parsed = tf.parse_single_example(
        serialized=serialized_example, features=features)

print(ex_parsed)  # -> prints a tensor

with tf.Session() as sess:
    print(sess.run([ex_parsed['seq']]))

I tried including tf.train.Coordinator() in the code but could not get that to work either.

Answer 1

The program hangs on the last line because you need to start queue runners before evaluating the output of a tf.TFRecordReader or a tf.train.string_input_producer() . Add a call to tf.train.start_queue_runners(sess) immediately after creating the session.

Alternatively, you can use the new tf.data API (in TensorFlow 1.4 or later; tf.contrib.data in TensorFlow 1.2 and 1.3) to read the data, without having to worry about queue runners:

# A `tf.data.Dataset` containing all of the records in the file named `fn`.
records = tf.data.TFRecordDataset(fn)

features = {'seq': tf.FixedLenFeature([], dtype=tf.int64)}

# A `tf.data.Dataset` whose elements are dictionaries mapping feature names
# (in this case 'seq') to tensors, based on `features`.
parsed = records.map(lambda x: tf.parse_single_example(x, features))

# Create a `tf.data.Iterator` to access individual elements of a `Dataset`. The
# system will take care of creating any background threads for you.
iterator = parsed.make_one_shot_iterator()

# `ex_parsed` represents the next element of the iterator. It is a dictionary
# mapping feature names to tensors.
ex_parsed = iterator.get_next()

with tf.Session() as sess:
    print(sess.run(ex_parsed['seq']))