[英]Packaging executable, shared library, and Python bindings not finding library
I have a project, cloudgen , that I would like to add bindings for Python so I can access some of the underlying functions.我有一个项目cloudgen ,我想为 Python 添加绑定,这样我就可以访问一些底层函数。 I have stubbed out the initial work on a branch .
我已经在一个分支上完成了最初的工作。 Because the main executable is built with cmake, I decided to use scikit-build to manage the build and use pybind11 to deal with the binding ( following this example repo ).
因为主要的可执行文件是用 cmake 构建的,所以我决定使用scikit-build来管理构建并使用pybind11来处理绑定(遵循这个示例 repo )。
When I run pip install.
当我运行
pip install.
in a virtual environment, everything appears to work as expected.在虚拟环境中,一切似乎都按预期工作。 I find the executable is installed to
<prefix>/bin
, the library goes into <prefix>/lib
, and the module goes into <prefix>/lib/pythonX.Y/site-packages/cloudgen
.我发现可执行文件安装到
<prefix>/bin
,库进入<prefix>/lib
,模块进入<prefix>/lib/pythonX.Y/site-packages/cloudgen
。 In fact, if I run pip uninstall cloudgen
, all of the correct files are uninstalled.事实上,如果我运行
pip uninstall cloudgen
,所有正确的文件都会被卸载。 However, my problems arise when I start to test the Python bindings.然而,当我开始测试 Python 绑定时,我的问题就出现了。 I find two separate but related problems.
我发现了两个独立但相关的问题。
python -m venv
, both the module and the executable are unable to resolve the path to the shared library.python -m venv
安装到虚拟环境中,则模块和可执行文件都无法解析共享库的路径。 Searching around, I came across this question which notes I could manipulate LD_LIBRARY_PATH
(or equivalently DYLD_LIBRARY_PATH
on macOS or PATH
on Windows), but that is normally frowned upon .环顾四周,我遇到了这个问题,其中指出我可以操纵
LD_LIBRARY_PATH
(或等效于 macOS 上的DYLD_LIBRARY_PATH
或 Windows 上的PATH
),但这通常是不赞成的。 That question references an open issue that refers to including additional build products (which as I said appears to not be my problem) but doesn't address the library path resolution.该问题引用了一个未解决的问题,该问题涉及包括其他构建产品(正如我所说,这似乎不是我的问题),但没有解决库路径解析。 I also came across this question asking about distributing the build products using scikit-build and this question using setuptools directly.
我还遇到了这个问题,询问有关使用 scikit-build 分发构建产品的问题,以及直接使用 setuptools 的问题。 Neither of the questions or answers address the library path resolution.
这些问题或答案都没有解决库路径解析问题。
My question is: What is the correct way to distribute a package that contains an executable, shared library, and Python binding module and have the path resolution Just Work™?我的问题是:分发包含可执行文件、共享库和 Python 绑定模块并具有路径解析 Just Work™ 的 package 的正确方法是什么?
A minimal working example is a bit much, but I created a gist to demonstrate the behavior.一个最小的工作示例有点多,但我创建了一个要点来演示该行为。
After a bit more digging (and carefully reading the CMake documentation on RPATH
), the correct answer appears to be explicitly setting RPATH
on installation.经过更多的挖掘(并仔细阅读
RPATH
上的 CMake 文档),正确的答案似乎是在安装时明确设置RPATH
。 The relevant change to the linked gist is to add the following to the CMakeLists.txt after creating the targets (adapted from the linked Wiki):对链接要点的相关更改是在创建目标后将以下内容添加到 CMakeLists.txt(改编自链接的 Wiki):
if (SKBUILD)
find_package(PythonExtensions REQUIRED)
set(lib_path "${PYTHON_PREFIX}/${CMAKE_INSTALL_LIBDIR}")
else()
set(lib_path "${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_LIBDIR}")
endif()
list(FIND CMAKE_PLATFORM_IMPLICIT_LINK_DIRECTORIES "${lib_path}" is_system)
if ("${is_system}" STREQUAL "-1")
set_target_properties(mwe.exe PROPERTIES
INSTALL_RPATH_USE_LINK_PATH TRUE
INSTALL_RPATH "${lib_path}")
# The following is necessary for installation in a virtual
# environment `python -m pip venv env`
set_target_properties(_mwe PROPERTIES
INSTALL_RPATH_USE_LINK_PATH TRUE
INSTALL_RPATH "${lib_path}")
endif()
This does require following the rest of the details about setting the RPATH
such as including (literally from the linked CMake Wiki):这确实需要遵循 rest 的有关设置
RPATH
的详细信息,例如包括(字面上来自链接的 CMake Wiki):
# use, i.e. don't skip the full RPATH for the build tree
set(CMAKE_SKIP_BUILD_RPATH FALSE)
# when building, don't use the install RPATH already
# (but later on when installing)
set(CMAKE_BUILD_WITH_INSTALL_RPATH FALSE)
set(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib")
# add the automatically determined parts of the RPATH
# which point to directories outside the build tree to the install RPATH
set(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE)
earlier in the CMakeLists.txt.在 CMakeLists.txt 的前面。 The net result is this should disable the dynamic searching of the library path (
LD_LIBRARY_PATH
or DYLD_LIBRARY_PATH
as appropriate) for the installed executable and Python module.最终结果是这应该禁用对已安装可执行文件和 Python 模块的库路径(适当的
LD_LIBRARY_PATH
或DYLD_LIBRARY_PATH
)的动态搜索。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.