通过 repl 导入很好，但会中断脚本

Question

I have the following test1.py script:

import tfrecorder

If I run it, it fails:

I have no name!@519b05bc3bf9:/test$ python test1.py 
2021-10-26 00:22:56.863971: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2021-10-26 00:22:56.863989: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Traceback (most recent call last):
  File "test1.py", line 1, in <module>
    import tfrecorder
  File "/usr/local/lib/python3.7/site-packages/tfrecorder/__init__.py", line 19, in <module>
    from tfrecorder import accessor
  File "/usr/local/lib/python3.7/site-packages/tfrecorder/accessor.py", line 28, in <module>
    from tfrecorder import converter
  File "/usr/local/lib/python3.7/site-packages/tfrecorder/converter.py", line 27, in <module>
    import apache_beam as beam
  File "/usr/local/lib/python3.7/site-packages/apache_beam/__init__.py", line 104, in <module>
    from apache_beam import coders
  File "/usr/local/lib/python3.7/site-packages/apache_beam/coders/__init__.py", line 19, in <module>
    from apache_beam.coders.coders import *
  File "/usr/local/lib/python3.7/site-packages/apache_beam/coders/coders.py", line 43, in <module>
    from future.moves import pickle
  File "/usr/local/lib/python3.7/site-packages/future/moves/__init__.py", line 8, in <module>
    import_top_level_modules()
  File "/usr/local/lib/python3.7/site-packages/future/standard_library/__init__.py", line 810, in import_top_level_modules
    with exclude_local_folder_imports(*TOP_LEVEL_MODULES):
  File "/usr/local/lib/python3.7/site-packages/future/standard_library/__init__.py", line 781, in __enter__
    module = __import__(m, level=0)
  File "/test/test.py", line 1, in <module>
    from tfrecorder import input_schema
  File "/usr/local/lib/python3.7/site-packages/tfrecorder/input_schema.py", line 26, in <module>
    from tfrecorder import types
  File "/usr/local/lib/python3.7/site-packages/tfrecorder/types.py", line 24, in <module>
    from tensorflow_transform import beam as tft_beam
  File "/usr/local/lib/python3.7/site-packages/tensorflow_transform/beam/__init__.py", line 20, in <module>
    from tensorflow_transform.beam import analyzer_cache
  File "/usr/local/lib/python3.7/site-packages/tensorflow_transform/beam/analyzer_cache.py", line 163, in <module>
    class _WriteToTFRecordGzip(beam.io.WriteToTFRecord):
AttributeError: module 'apache_beam' has no attribute 'io'

However, in the REPL it's fine:

I have no name!@ff017054aef4:/$ python
Python 3.7.12 (default, Oct 12 2021, 03:26:18) 
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tfrecorder
2021-10-26 00:24:33.293181: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2021-10-26 00:24:33.293201: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
>>> 

It doesn't make any sense for me. Any ideas?

Answer 1

It appears that this is a bug, and that has been rectified in the latest version of apache_beam . The explanation in this thread is descriptive. Circular Import Issue in apache_beam

The problem is that TOP_LEVEL_MODULES variable defines a list of modules which includes test . Your test module's name collides with the built in module in apache_beam. You can try renaming your test module, and check.

You don't see the same issue in shell because sys.path does not have the directory /test In the shell environment.