堆栈内存溢出
  简体   繁体   English

如何修复符号查找错误：群集环境中的未定义符号错误

[英]How to fix symbol lookup error: undefined symbol errors in a cluster environment

 原文  2014-11-21 05:32:18  4   2    linux/ linker/ cluster-computing/ python-2.6/ gdal

问题描述

I'm working on some python code that extracts some image data from an ECW file using GDAL ( http://www.gdal.org/ ) and its python bindings. 我正在研究一些python代码，它使用GDAL（ http://www.gdal.org/ ）及其python绑定从ECW文件中提取一些图像数据。 GDAL was built from source to have ECW support. GDAL是从源代码构建的，以获得ECW支持。

The program is run on a cluster server that I ssh into. 该程序在我进入的集群服务器上运行。 I have tested the program through the ssh terminal and it runs fine. 我已经通过ssh终端测试了程序，运行正常。 However, I would now like to submit a job to the cluster using qsub, but it reports the following: 但是，我现在想使用qsub向集群提交作业，但它报告以下内容： 

Traceback (most recent call last):
  File "./gdal-test.py", line 5, in <module>
    from osgeo import gdal
  File "/home/h3/ctargett/.local/lib/python2.6/site-packages/GDAL-1.11.1-py2.6-linux-x86_64.egg/osgeo/__init__.py", line 21, in <module>
    _gdal = swig_import_helper()
  File "/home/h3/ctargett/.local/lib/python2.6/site-packages/GDAL-1.11.1-py2.6-linux-x86_64.egg/osgeo/__init__.py", line 17, in swig_import_helper
    _mod = imp.load_module('_gdal', fp, pathname, description)
ImportError: /mnt/aeropix/prgs/.local/lib/libgdal.so.1: undefined symbol: H5Eset_auto2

I did a bit more digging and tried using LD_DEBUG=symbols to try and work out where the difference was, but that's about as far as my knowledge/understanding has got me. 我做了一些挖掘并尝试使用LD_DEBUG=symbols来尝试找出差异所在的位置，但这就是我的知识/理解所带来的。

For reference, here's what happens with LD_DEBUG=symbols and running the code in the ssh terminal (piping through grep H5Eset_auto2 to reduce some of the output): 作为参考，这是LD_DEBUG=symbols和在ssh终端中运行代码时发生的事情（通过grep H5Eset_auto2管道以减少一些输出）：

Symbol debug output for code running in ssh terminal: 在ssh终端中运行的代码的符号调试输出： 

 11359: symbol=H5Eset_auto2;  lookup in file=/usr/bin/python26 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/usr/lib64/libpython2.6.so.1.0 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libpthread.so.0 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libdl.so.2 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libutil.so.1 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libm.so.6 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libc.so.6 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/ld-linux-x86-64.so.2 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/home/h3/ctargett/.local/lib/python2.6/site-packages/GDAL-1.11.1-py2.6-linux-x86_64.egg/osgeo/_gdal.so [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/usr/lib64/libpython2.6.so.1.0 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/mnt/aeropix/prgs/.local/lib/libgdal.so.1 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/usr/lib64/libstdc++.so.6 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libm.so.6 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libgcc_s.so.1 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libpthread.so.0 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libc.so.6 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libdl.so.2 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libutil.so.1 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/mnt/aeropix/prgs/.local/lib/libhdf5.so.7 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/usr/bin/python26 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/usr/lib64/libpython2.6.so.1.0 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libpthread.so.0 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libdl.so.2 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libutil.so.1 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libm.so.6 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libc.so.6 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/ld-linux-x86-64.so.2 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/home/h3/ctargett/.local/lib/python2.6/site-packages/GDAL-1.11.1-py2.6-linux-x86_64.egg/osgeo/_gdal.so [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/usr/lib64/libpython2.6.so.1.0 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/mnt/aeropix/prgs/.local/lib/libgdal.so.1 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/usr/lib64/libstdc++.so.6 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libm.so.6 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libgcc_s.so.1 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libpthread.so.0 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libc.so.6 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libdl.so.2 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/lib64/libutil.so.1 [0]
 11359: symbol=H5Eset_auto2;  lookup in file=/mnt/aeropix/prgs/.local/lib/libhdf5.so.7 [0]

Symbol debug output for code submitted using qsub: 使用qsub提交的代码的符号调试输出： 

 16915: symbol=H5Eset_auto2;  lookup in file=/usr/bin/python26 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/usr/lib64/libpython2.6.so.1.0 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/libpthread.so.0 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/libdl.so.2 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/libutil.so.1 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/libm.so.6 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/libc.so.6 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/ld-linux-x86-64.so.2 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/home/h3/ctargett/.local/lib/python2.6/site-packages/GDAL-1.11.1-py2.6-linux-x86_64.egg/osgeo/_gdal.so [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/usr/lib64/libpython2.6.so.1.0 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/mnt/aeropix/prgs/.local/lib/libgdal.so.1 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/usr/lib64/libstdc++.so.6 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/libm.so.6 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/libgcc_s.so.1 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/libpthread.so.0 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/libc.so.6 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/libdl.so.2 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/libutil.so.1 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/mnt/aeropix/prgs/.local/lib/libhdf5.so.7 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/usr/lib64/libjpeg.so.62 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/usr/lib64/libpng12.so.0 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/usr/lib64/libpq.so.4 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/usr/lib64/libcurl.so.3 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/usr/lib64/libgssapi_krb5.so.2 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/usr/lib64/libkrb5.so.3 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/usr/lib64/libk5crypto.so.3 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/libcom_err.so.2 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/usr/lib64/libidn.so.11 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/libssl.so.6 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/libcrypto.so.6 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/mnt/aeropix/prgs/.local/lib/libNCSEcw.so.0 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/mnt/aeropix/prgs/.local/lib/libNCSEcwC.so.0 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/mnt/aeropix/prgs/.local/lib/libNCSCnet.so.0 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/mnt/aeropix/prgs/.local/lib/libNCSUtil.so.0 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/librt.so.1 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/usr/lib64/libxml2.so.2 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/mnt/aeropix/prgs/.local/lib/libz.so.1 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/ld-linux-x86-64.so.2 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/libcrypt.so.1 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/libresolv.so.2 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/libnsl.so.1 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/usr/lib64/libkrb5support.so.0 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/libkeyutils.so.1 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/libselinux.so.1 [0]
 16915: symbol=H5Eset_auto2;  lookup in file=/lib64/libsepol.so.1 [0]
 16915: /mnt/aeropix/prgs/.local/lib/libgdal.so.1: error: symbol lookup error: undefined symbol: H5Eset_auto2 (fatal)
ImportError: /mnt/aeropix/prgs/.local/lib/libgdal.so.1: undefined symbol: H5Eset_auto2

I guess I'm not sure why it seems to stop looking in libgdal.so.1 when submitted using qsub, when it continues to look when just run in the terminal. 我想我不知道为什么它似乎停止在使用qsub提交时查看libgdal.so.1，当它继续看起来只是在终端中运行时。 I also note that the qsub job is able to correctly locate libhdf5.so.7 (which is where it should find H5Eset_auto2 ) as it can find a different symbol, H5Eprint : 我还注意到qsub作业能够正确定位libhdf5.so.7 （它应该找到H5Eset_auto2 ），因为它可以找到一个不同的符号， H5Eprint ： 

 16915: symbol=H5Eprint;  lookup in file=/usr/lib64/libpython2.6.so.1.0 [0]
 16915: symbol=H5Eprint;  lookup in file=/mnt/aeropix/prgs/.local/lib/libgdal.so.1 [0]
 16915: symbol=H5Eprint;  lookup in file=/usr/lib64/libstdc++.so.6 [0]
 16915: symbol=H5Eprint;  lookup in file=/lib64/libm.so.6 [0]
 16915: symbol=H5Eprint;  lookup in file=/lib64/libgcc_s.so.1 [0]
 16915: symbol=H5Eprint;  lookup in file=/lib64/libpthread.so.0 [0]
 16915: symbol=H5Eprint;  lookup in file=/lib64/libc.so.6 [0]
 16915: symbol=H5Eprint;  lookup in file=/lib64/libdl.so.2 [0]
 16915: symbol=H5Eprint;  lookup in file=/lib64/libutil.so.1 [0]
 16915: symbol=H5Eprint;  lookup in file=/mnt/aeropix/prgs/.local/lib/libhdf5.so.7 [0]

Any pointers on this would be incredibly useful at this stage (I hope that's enough information - I'm more than happy to provide more information, I'm just not sure what else might be useful at this stage). 关于这一点的任何指针在这个阶段都非常有用（我希望这些信息足够多 - 我很乐意提供更多信息，我只是不确定在这个阶段还有什么用处）。

EDIT: 编辑：

It seems that the contents of /usr/bin are different for jobs submitted using qsub (specifically libtool is missing). 对于使用qsub提交的作业（特别是缺少libtool ）， /usr/bin的内容似乎不同。 This is being investigated. 这正在调查中。

2 个解决方案

解决方案1
 23 已采纳  2014-11-24 23:30:25

After two dozens of comments to understand the situation, it was found that the libhdf5.so.7 was actually a symlink (with several levels of indirection) to a file that was not shared between the queued processes and the interactive processes. 经过两十条评论来了解情况后，发现libhdf5.so.7实际上是一个符号链接 （具有多个间接级别）到一个未在排队进程和交互式进程之间共享的文件 。 This means even though the symlink itself lies on a shared filesystem, the contents of the file do not and as a result the process was seeing different versions of the library. 这意味着即使符号链接本身位于共享文件系统上，文件的内容也不会，因此进程看到了库的不同版本。

For future reference: other than checking LD_LIBRARY_PATH , it's always a good idea to check a library with nm -D to see if the symbols actually exist. 供将来参考：除了检查LD_LIBRARY_PATH ，检查带有nm -D的库总是一个好主意，看看符号是否确实存在。 In this case it was found that they do exist in interactive mode but not when run in the queue. 在这种情况下，发现它们确实以交互模式存在，但在队列中运行时不存在。 A quick md5sum revealed that the files were actually different. 一个快速的md5sum显示文件实际上是不同的。

解决方案2
 0  2017-11-30 11:10:38

yum update

helped me out. 帮帮我了 After I had 我之后 

wget: symbol lookup error: wget: undefined symbol: psl_latest

暂无
暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 imp：符号查找错误：未定义符号sldext - imp : symbol lookup error : undefined symbol sldext 符号查找错误：未定义的符号：_alloca - symbol lookup error: undefined symbol: _alloca java：符号查找错误：java：未定义的符号：conda环境中的JLI_StringDup - java: symbol lookup error: java: undefined symbol: JLI_StringDup in conda environment 如何修复未定义符号的genymotion错误：drmGetDevice？ - How to fix genymotion error of undefined symbol : drmGetDevice? 符号查找错误未定义符号：mysql_init - Symbol lookup error undefined symbol: mysql_init 系统查找错误未定义符号SFML - system lookup error undefined symbol SFML 符号查找错误：未定义符号：FT_Get_Font_Format - symbol lookup error: undefined symbol: FT_Get_Font_Format 符号查找错误未定义符号，但似乎所有符号都存在 - Symbol lookup error undefined symbol, but all symbols seem to be present 符号查找错误：未定义的符号 atexit (linux/C) - Symbol lookup error: undefined symbol atexit (linux/C) 如何修复错误：无法加载共享库，符号未定义 - How to fix the error: Cannot load shared library , symbol undefined
相关标签
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM