AWS Glue：在 AWS Glue python shell 作業 Python 庫路徑上安裝 pandas==1.0.3

Question

我有需要使用Explode 和 json_normalize 的pandas 數據框屬性的要求。 默認情況下，python 膠水外殼似乎運行 0.24.2 熊貓版本。

我能夠使用 .whl 版本 pandas-0.23.0-cp36-cp36m-manylinux1_x86_64.whl。當我嘗試通過 pandas-0.25.3-cp35-cp35m-manylinux1_x86_64.whl,pandas-1.0 為 pandas 版本提供 .whl 文件時.0-cp38-cp38-manylinux1_x86_64.whl,pandas-1.0.3-cp38-cp38-manylinux1_x86_64.whl 都無法加載，並顯示以下錯誤消息：

  Traceback (most recent call last):
  File "/glue/lib/installation/pandas/__init__.py", line 32, in <module>
    from pandas._libs import hashtable as _hashtable, lib as _lib, tslib as _tslib
  File "/glue/lib/installation/pandas/_libs/__init__.py", line 3, in <module>
    from .tslibs import (
  File "/glue/lib/installation/pandas/_libs/tslibs/__init__.py", line 3, in <module>
    from .conversion import localize_pydatetime, normalize_date
ModuleNotFoundError: No module named 'pandas._libs.tslibs.conversion'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/runscript.py", line 211, in <module>
    runpy.run_path(temp_file_path, run_name='__main__')
  File "/usr/local/lib/python3.6/runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "/usr/local/lib/python3.6/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmp/glue-python-scripts-maew3inn/EPP_Json_To_CSV.py", line 1, in <module>
  File "/glue/lib/installation/pandas/__init__.py", line 37, in <module>
    f"C extension: {module} not built. If you want to import "
ImportError: C extension: No module named 'pandas._libs.tslibs.conversion' not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/runscript.py", line 230, in <module>
    raise e_type(e_value).with_traceback(new_stack)
  File "/tmp/glue-python-scripts-maew3inn/EPP_Json_To_CSV.py", line 1, in <module>
  File "/glue/lib/installation/pandas/__init__.py", line 37, in <module>
    f"C extension: {module} not built. If you want to import "
ImportError: C extension: No module named 'pandas._libs.tslibs.conversion' not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.

Answer 1

目前，您無法將 pandas 庫導入 Glue。 AWS GLUE 文檔

只能使用純 Python 庫。 尚不支持依賴 C 擴展的庫，例如 pandas Python 數據分析庫。

這是一個重復的問題： Use AWS Glue Python with NumPy and Pandas Python Packages