堆栈内存溢出
  简体   繁体   English

交叉编译python ModuleNotFoundError：模块信息来自哪里？

[英]cross-compile python ModuleNotFoundError: where the module info is from?

 原文  2020-11-19 18:52:35  2   1    python/ python-3.x/ cross-compiling/ buildroot

问题描述

I'm cross-building python 3.8.我正在交叉构建python 3.8。 Two errors look simiar:两个错误看起来相似：

$ python3 -m venv .
Error: Command '['/home/p3t/penv386/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

$ bin/python3 -Im ensurepip --upgrade --default-pip
Traceback (most recent call last):
  File "<frozen zipimport>", line 520, in _get_decompress_func
ModuleNotFoundError: No module named 'zlib'

$ pip3 install --user pipenv
Traceback (most recent call last):
  File "/usr/bin/pip3", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 35, in <module>
  File "/usr/lib/python3.8/plistlib.py", line 65, in <module>
ModuleNotFoundError: No module named 'xml'

I've seen a few similar threads on SO:我在 SO 上看到了一些类似的主题：

  • build python with zlib : Edit Module/Setup to include zlib or install zlib. 使用 zlib 构建 python ：编辑模块/设置以包含 zlib 或安装 zlib。 I did the edit and verified libz exists.我进行了编辑并验证了 libz 存在。 It also has a python snippet to verify but should be run if import ok.它也有一个 python 片段来验证，但如果导入正常，应该运行。
  • no module named zlib : Similar solution. 没有名为 zlib 的模块：类似的解决方案。
  • A few other threads are basically very similar to the above two.其他几个线程与上面两个基本非常相似。

Since the "/usr/lib/libz*" does exist and the xml lib "/usr/lib/libexpat*" does exists too, I start to think the module info probably is missing.由于“/usr/lib/libz*”确实存在并且xml lib“/usr/lib/libexpat*”也确实存在，我开始认为模块信息可能丢失了。 How to make sure those two modules are properly populating with their information in the python build?如何确保这两个模块在 python 构建中正确填充了它们的信息？

I cross-compiled using buildroot 2020.08.2.我使用 buildroot 2020.08.2 交叉编译。 From the buildroot configuration, both xml and zlib are selected.从 buildroot 配置中，选择了 xml 和 zlib。 I did a strace, have not seen what could be the cause, here for the venv:我做了一个 strace，还没有看到可能是什么原因，这里是 venv：

$ cat -n log2-venv | egrep '(openat|newfstatat)' | grep '\.so'
     5  5835  openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = -1 EACCES (Permission denied)
     6  5835  openat(AT_FDCWD, "/lib64/tls/aarch64/libpython3.8.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
     8  5835  openat(AT_FDCWD, "/lib64/tls/libpython3.8.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
    10  5835  openat(AT_FDCWD, "/lib64/aarch64/libpython3.8.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
    12  5835  openat(AT_FDCWD, "/lib64/libpython3.8.so.1.0", O_RDONLY|O_CLOEXEC) = 3

    20  5835  openat(AT_FDCWD, "/lib64/libcrypt.so.1", O_RDONLY|O_CLOEXEC) = 3
    28  5835  openat(AT_FDCWD, "/lib64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
    36  5835  openat(AT_FDCWD, "/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
    43  5835  openat(AT_FDCWD, "/lib64/libutil.so.1", O_RDONLY|O_CLOEXEC) = 3
    50  5835  openat(AT_FDCWD, "/lib64/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
    58  5835  openat(AT_FDCWD, "/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3

   123  5835  newfstatat(AT_FDCWD, "/usr/lib/python3.8/encodings/__init__.cpython-38-aarch64-linux-gnu.so", 0x7fc2ad5c10, 0) = -1 ENOENT (No such file or directory)
   124  5835  newfstatat(AT_FDCWD, "/usr/lib/python3.8/encodings/__init__.abi3.so", 0x7fc2ad5c10, 0) = -1 ENOENT (No such file or directory)
   125  5835  newfstatat(AT_FDCWD, "/usr/lib/python3.8/encodings/__init__.so", 0x7fc2ad5c10, 0) = -1 ENOENT (No such file or directory)

   496  5835  newfstatat(AT_FDCWD, "/usr/lib/python3.8/collections/__init__.cpython-38-aarch64-linux-gnu.so", 0x7fc2ad43a0, 0) = -1 ENOENT (No such file or directory)
   497  5835  newfstatat(AT_FDCWD, "/usr/lib/python3.8/collections/__init__.abi3.so", 0x7fc2ad43a0, 0) = -1 ENOENT (No such file or directory)
   498  5835  newfstatat(AT_FDCWD, "/usr/lib/python3.8/collections/__init__.so", 0x7fc2ad43a0, 0) = -1 ENOENT (No such file or directory)

   544  5835  newfstatat(AT_FDCWD, "/usr/lib/python3.8/lib-dynload/_heapq.cpython-38-aarch64-linux-gnu.so", {st_mode=S_IFREG|0755, st_size=18528, ...}, 0) = 0
   546  5835  openat(AT_FDCWD, "/usr/lib/python3.8/lib-dynload/_heapq.cpython-38-aarch64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3

   578  5835  newfstatat(AT_FDCWD, "/usr/lib/python3.8/site-packages/pkg_resources/__init__.cpython-38-aarch64-linux-gnu.so", 0x7fc2ad60c0, 0) = -1 ENOENT (No such file or directory)
   579  5835  newfstatat(AT_FDCWD, "/usr/lib/python3.8/site-packages/pkg_resources/__init__.abi3.so", 0x7fc2ad60c0, 0) = -1 ENOENT (No such file or directory)
   580  5835  newfstatat(AT_FDCWD, "/usr/lib/python3.8/site-packages/pkg_resources/__init__.so", 0x7fc2ad60c0, 0) = -1 ENOENT (No such file or directory)

   615  5835  newfstatat(AT_FDCWD, "/usr/lib/python3.8/lib-dynload/binascii.cpython-38-aarch64-linux-gnu.so", {st_mode=S_IFREG|0755, st_size=27656, ...}, 0) = 0
   617  5835  openat(AT_FDCWD, "/usr/lib/python3.8/lib-dynload/binascii.cpython-38-aarch64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3

   629  5835  newfstatat(AT_FDCWD, "/usr/lib/python3.8/importlib/__init__.cpython-38-aarch64-linux-gnu.so", 0x7fc2ad3af0, 0) = -1 ENOENT (No such file or directory)
   630  5835  newfstatat(AT_FDCWD, "/usr/lib/python3.8/importlib/__init__.abi3.so", 0x7fc2ad3af0, 0) = -1 ENOENT (No such file or directory)
   631  5835  newfstatat(AT_FDCWD, "/usr/lib/python3.8/importlib/__init__.so", 0x7fc2ad3af0, 0) = -1 ENOENT (No such file or directory)

   766  5835  newfstatat(AT_FDCWD, "/usr/bin/lzma/__init__.cpython-38-aarch64-linux-gnu.so", 0x7fc2ad3510, 0) = -1 ENOTDIR (Not a directory)
   767  5835  newfstatat(AT_FDCWD, "/usr/bin/lzma/__init__.abi3.so", 0x7fc2ad3510, 0) = -1 ENOTDIR (Not a directory)
   768  5835  newfstatat(AT_FDCWD, "/usr/bin/lzma/__init__.so", 0x7fc2ad3510, 0) = -1 ENOTDIR (Not a directory)

   788  5835  newfstatat(AT_FDCWD, "/usr/lib/python3.8/lib-dynload/grp.cpython-38-aarch64-linux-gnu.so", {st_mode=S_IFREG|0755, st_size=14784, ...}, 0) = 0
   789  5835  openat(AT_FDCWD, "/usr/lib/python3.8/lib-dynload/grp.cpython-38-aarch64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3
   810  5835  newfstatat(AT_FDCWD, "/usr/lib/python3.8/lib-dynload/_struct.cpython-38-aarch64-linux-gnu.so", {st_mode=S_IFREG|0755, st_size=38352, ...}, 0) = 0
   811  5835  openat(AT_FDCWD, "/usr/lib/python3.8/lib-dynload/_struct.cpython-38-aarch64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3

   838  5835  newfstatat(AT_FDCWD, "/usr/bin/lzma/__init__.cpython-38-aarch64-linux-gnu.so", 0x7fc2ad43a0, 0) = -1 ENOTDIR (Not a directory)
   839  5835  newfstatat(AT_FDCWD, "/usr/bin/lzma/__init__.abi3.so", 0x7fc2ad43a0, 0) = -1 ENOTDIR (Not a directory)
   840  5835  newfstatat(AT_FDCWD, "/usr/bin/lzma/__init__.so", 0x7fc2ad43a0, 0) = -1 ENOTDIR (Not a directory)

   918  5835  newfstatat(AT_FDCWD, "/usr/lib/python3.8/lib-dynload/math.cpython-38-aarch64-linux-gnu.so", {st_mode=S_IFREG|0755, st_size=45240, ...}, 0) = 0
   919  5835  openat(AT_FDCWD, "/usr/lib/python3.8/lib-dynload/math.cpython-38-aarch64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3
   930  5835  newfstatat(AT_FDCWD, "/usr/lib/python3.8/lib-dynload/_datetime.cpython-38-aarch64-linux-gnu.so", {st_mode=S_IFREG|0755, st_size=75320, ...}, 0) = 0
   931  5835  openat(AT_FDCWD, "/usr/lib/python3.8/lib-dynload/_datetime.cpython-38-aarch64-linux-gnu.so", O_RDONLY|O_CLOEXEC) = 3

1 个解决方案

解决方案1
 0  2020-11-20 15:31:23

Not sure exactly, it might be a bug in buildroot that does not resolve dependencies for this scenario.不确定，这可能是 buildroot 中的一个错误，无法解决此场景的依赖关系。 A deeper tracking of the source code seems to show: The xml and zlib packages are enabled when in configuration the python-pip is selected.对源代码的更深入跟踪似乎显示：在配置中选择了 python-pip 时，将启用 xml 和 zlib 包。 However, I guess, the module info is compiled by the host-python into .pyc files and included into the target python installation.但是，我猜想，模块信息由 host-python 编译成 .pyc 文件并包含在目标 python 安装中。 Buildroot does not rebuild host-python. Buildroot 不会重建 host-python。 An implication of that is the module info missing from the target python.这意味着目标 python 中缺少模块信息。

Based on the reasoning, this solution looks to work for me:基于推理，这个解决方案看起来对我有用：

  • After enabling python-pip in configuration, run ls output/build/ | grep python在配置中启用 python-pip 后，运行ls output/build/ | grep python ls output/build/ | grep python to find all the python related packages. ls output/build/ | grep python查找所有与 python 相关的包。
  • On each of those packages, run a make *-dirclean on them.在每个包上，对它们运行 make *-dirclean 。 For example, if only python3-3.8.6 and host-python3-3.8.6 were found, run the commands below:例如，如果只python3-3.8.6和host-python3-3.8.6 ，则运行以下命令：
make host-python3-dirclean
make python3-dirclean
  • Run make again to rebuild.再次运行make以重建。

No need to modify Module/Setup or Module/Setup.local .无需修改Module/Setup或Module/Setup.local 。

暂无
暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 如何交叉编译Python？ - How to cross-compile Python? 带有Python支持的gdb在交叉编译开发中 - gdb with Python support in cross-compile development Linux for Windows 上的交叉编译扩展 - Cross-compile extension on Linux for Windows Python 从带有“ModuleNotFoundError”的模块导入 - Python import from a module with “ModuleNotFoundError” 尝试使用distutils为mingw32交叉编译SWIG Python扩展时出错 - Error when trying to cross-compile SWIG Python extension for mingw32 using distutils 无法为 x86、uclibc 交叉编译 Python-2.7.18 - Unable to cross-compile Python-2.7.18 for x86,uclibc python 交叉编译中 ac_cv_file 标志的含义是什么 - What's meaning of ac_cv_file flags in python cross-compile “ ModuleNotFoundError”。 如何获取我的python以确定模块的安装位置？ - “ModuleNotFoundError”. How to get my python to identify where the module is installed? 自定义模块的 Python ModuleNotFoundError - Python ModuleNotFoundError for Custom Module Python ModuleNotFoundError：没有模块命名 - Python ModuleNotFoundError: No module named
相关标签
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM