“SystemError: error return without exception set” when using tf.data.Dataset.from_generator

Question

I am hitting a strange runtime error when using tf.data.Dataset.from_generator .

import tensorflow as tf

def gen():
  while True:
    yield 0, 0

batch = (
  tf.data.Dataset.from_generator(
    gen, output_types=(tf.int32, tf.int32))
  .batch(8)
  .repeat()
  .make_one_shot_iterator()
  .get_next()
)

sess = tf.InteractiveSession()
sess.run(batch)

would print messages like this

Exception ignored in: <generator object _yield_value at 0x000001FD9ED6A7D8>
Traceback (most recent call last):
  File "C:\Users\user209974\AppData\Local\Continuum\miniconda3\envs\tf\lib\site-packages\tensorflow\python\data\util\nest.py", line 100, in _yield_value
    yield value
SystemError: error return without exception set

The error is thrown at runtime when calling Session.run , apparently each time the generator is called. What's weird though is that the returned values seems to be correct despite this error being thrown.

So what is causing this error and how to make it away? (For some reason, removing batching or returning a single value in gen makes the error go away).

EDIT

I realized that those errors show up when running on Debug mode under PyCharm. If I run the code in the python console, as a script, or in PyCharm but not in Debug mode, those messages are not printed.

Answer 1

It seems like a Pycharm problem. The solution is setting this environment variable in pycharm settings:

PYDEVD_USE_FRAME_EVAL=NO

Check this post that offers more information