[英]Python3 shared extension doesn't link against library dependency
I'm creating a shared Python extension for my library and I'm using distutils
to build it.我正在为我的库创建一个共享的 Python 扩展,我正在使用distutils
来构建它。
These are the relevant sections of my setup.py
:这些是我的setup.py
的相关部分:
import distuitls.core as dc
from os.path import join as path_join
module = dc.Extension(module_name,
sources = [path_join(meson_src_root, "py3_bindings", "module.c")],
include_dirs = [path_join(meson_src_root, "include")],
libraries = ["bbmputil"],
runtime_library_dirs = [meson_build_root])
dc.setup(name = module_name,
version = module_version,
description = "Python3 bindings for the bbmp_utils library",
ext_modules = [module])
Running $ setup.py build
results in the shared extension module being built successfully, but it isn't getting linked against the "bbmputil"
library.运行$ setup.py build
导致成功$ setup.py build
共享扩展模块,但它没有链接到"bbmputil"
库。
$ ldd build/lib.linux-x86_64-3.8/bbmp_utils.cpython-38-x86_64-linux-gnu.so
linux-vdso.so.1 (0x00007ffc85ce1000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007f49f0d70000)
/usr/lib64/ld-linux-x86-64.so.2 (0x00007f49f0f74000)
libbbmputil.so
is nowhere to be found, despite being specified in the libraries
kwarg of Extension()
. libbbmputil.so
是无处可寻,尽管在被指定的libraries
kwarg的Extension()
It does exist in the location specified in the runtime_library_dirs
kwarg.它确实存在于runtime_library_dirs
kwarg 中指定的位置。
This leads to the python interpreter raising an ImportError
exception when a symbol from the non-linked library is referenced in the extension:当扩展中引用来自非链接库的符号时,这会导致 python 解释器引发ImportError
异常:
$ env PYTHONPATH="sharedextension_build_path" python3
>>> import bbmp_utils
ImportError: /home/bogdan/dev/bbmp_utils/build_dbg/build/lib.linux-x86_64-3.8/bbmp_utils.cpython-38-x86_64-linux-gnu.so: undefined symbol: bbmp_vertflip
where bbmp_vertflip
is a symbol defined in the library that doesn't seem to be linked for some reason.其中bbmp_vertflip
是库中定义的符号,由于某种原因似乎没有链接。
The two C compiler invocations look as follows:两个 C 编译器调用如下所示:
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fPIC -I/home/bogdan/dev/bbmp_utils/include -I/usr/include/python3.8 -c /home/bogdan/dev/bbmp_utils/py3_bindings/module.c -o build/temp.linux-x86_64-3.8/home/bogdan/dev/bbmp_utils/py3_bindings/module.o
gcc -pthread -shared -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now build/temp.linux-x86_64-3.8/home/bogdan/dev/bbmp_utils/py3_bindings/module.o -L/usr/lib -Wl,--enable-new-dtags,-R/home/bogdan/dev/bbmp_utils/build_dbg -lbbmputil -o build/lib.linux-x86_64-3.8/bbmp_utils.cpython-38-x86_64-linux-gnu.so
In the 2nd invocation both -lbbmputil
as well as -R
are passed properly when building the shared extension so I'm out of ideas.在第二次调用中, -lbbmputil
和-R
在构建共享扩展时都被正确传递,所以我没有想法。
Attempting to build a module that utilizes functions and other symbols from the math shared library:尝试构建一个使用数学共享库中的函数和其他符号的模块:
#!/usr/bin/env python3
import distutils.core as dc
module = dc.Extension('example',
sources = ['example.c'],
libraries = ['m'])
dc.setup(name = 'example',
version = '0.1',
ext_modules = [module])
$ ./setup.py build
$ ldd .../.../example.cpython-38-x86_64-linux-gnu.so
linux-vdso.so.1 (0x00007ffd0b9e5000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007fab528e8000)
/usr/lib64/ld-linux-x86-64.so.2 (0x00007fab52aec000)
Again, libm.so
dependency is nowhere to be found.同样,找不到libm.so
依赖项。
Environment:环境:
UPDATE : The problem in this case is the linker optimization option
--as-needed
that is enabled by default , see Missing a library in ldd after using gcc -l更新:这种情况下的问题是链接器优化选项--as-needed
默认启用,请参阅使用 gcc -l 后在 ldd 中丢失库Adding
--no-as-needed
fixes this error添加--no-as-needed
可修复此错误
For debugging linker errors you can use LD_DEBUG=files,libs /usr/local/ABC/bin/ABC
where ABC
is the executable that throws linker errors at runtime, cf http://www.bnikolic.co.uk/blog/linux-ld-debug.html and libm.so.6: cannot open shared object file: No such file or directory On linux you locate a .so
with ie locate libm
(i think you know this)对于调试链接器错误,您可以使用LD_DEBUG=files,libs /usr/local/ABC/bin/ABC
其中ABC
是在运行时引发链接器错误的可执行文件,参见http://www.bnikolic.co.uk/blog/linux -ld-debug.html和下的libm.so.6:无法打开共享对象文件:没有这样的文件或目录在Linux上您找到一个.so
用即locate libm
(我想你知道这一点)
As the linking is dynamically it is an option to specify the path where your .so
files can be found using the library_dirs
option of disutils.core
that is the -L
or equivalently LD_LIBRARY_PATH
gcc linker option and for reasons of debugging and testing i would use the absolute path ( https://docs.python.org/2/distutils/apiref.html )由于链接是动态的,因此可以选择使用disutils.core
的library_dirs
选项指定可以找到.so
文件的路径,该选项是-L
或等效的LD_LIBRARY_PATH
gcc 链接器选项,出于调试和测试的原因,我将使用绝对路径( https://docs.python.org/2/distutils/apiref.html )
In your python minimal example the code is then :在你的 python 最小示例中,代码是:
#!/usr/bin/env python3
import distutils.core as dc
module = dc.Extension('example',
sources = ['example.c'],
library_dirs = ['/usr/lib/x86_64-linux-gnu/libm.so'],
libraries = ['m'])
dc.setup(name = 'example',
version = '0.1',
ext_modules = [module])
You use the -R
linker flag to specify the rpath
in your gcc invokation, cf Shared library dependencies with distutils and What does the gcc -R parameter do?您使用-R
链接器标志在 gcc 调用中指定rpath
,cf Shared library dependencies with distutils以及gcc -R 参数有什么作用? . . In https://www.mpcdf.mpg.de/services/computing/software/libraries/static-and-dynamic-linking-on-linux-systems is a description of the linking process.在https://www.mpcdf.mpg.de/services/computing/software/libraries/static-and-dynamic-linking-on-linux-systems 中是对链接过程的描述。 It is said that LD_LIBRARY_PATH
or equivalently the -L
gcc linker option overrides the rpath and that it should be avoided, however you should give it a try anyway ...据说LD_LIBRARY_PATH
或等效的-L
gcc 链接器选项会覆盖 rpath 并且应该避免它,但是无论如何你应该尝试一下......
Another possiblity for this behavior could be permission problems, ie when you execute example
does it have the permission to access libm
cf https://unix.stackexchange.com/questions/303292/permission-denied-on-some-shared-libraries这种行为的另一种可能性可能是权限问题,即当您执行example
,它是否有权访问libm
cf https://unix.stackexchange.com/questions/303292/permission-denied-on-some-shared-libraries
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.