Importing Python module from .so file fails on target system with lower GLIBC version

Question

I have built a shared library ('libFoo.so') using GCC 4.9.0 (built from source) on Ubuntu 14.04 and built a Boost.Python wrapper library around it ('Foo.so' for the module Foo, as per naming convention).

For distribution purposes, I am including all shared object dependencies in the same directory, which I extracted using ldd:

machine1:~/lib$ ls
Foo.so    libc.so.6   libgcc_s.so.1     libFoo.so  libpthread.so.0      librt.so.1      libutil.so.1
libboost_python.so.1.55.0  libdl.so.2  libm.so.6       libpython2.7.so.1.0  libstdc++.so.6  libz.so.1

machine1:~/lib$ ldd Foo.so 
    linux-vdso.so.1 =>  (0x00007fff319fe000)
    libboost_python.so.1.55.0 (0x00007f9d568aa000)
    libpython2.7.so.1.0 (0x00007f9d56342000)
    libFoo.so (0x00007f9d55e92000)
    libm.so.6 (0x00007f9d55b8c000)
    libc.so.6 (0x00007f9d557c5000)
    libutil.so.1 (0x00007f9d555c2000)
    libpthread.so.0 (0x00007f9d553a4000)
    libdl.so.2 (0x00007f9d5519f000)
    librt.so.1 (0x00007f9d54f97000)
    libstdc++.so.6 => /usr/local/lib64/libstdc++.so.6 (0x00007f9d54c8d000)
    libgcc_s.so.1 => /usr/local/lib64/libgcc_s.so.1 (0x00007f9d54a76000)
    libz.so.1 (0x00007f9d5485d000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f9d56d82000)

On the original system, I can now just write:

machine1:~/lib$ LD_LIBRARY_PATH=. PYTHONPATH=. python
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import Foo
>>>

However, on a different system (Ubuntu 12.04 LTS), I am having serious problems getting this step to work.

machine2:~/lib$ LD_LIBRARY_PATH=. PYTHONPATH=. python
Inconsistency detected by ld.so: dl-close.c: 759: _dl_close: Assertion `map->l_init_called' failed!

Ok, there is some sort of conflict even starting Python, so let's move the offending libraries somewhere else:

machine2:~/lib$ mv libc.so.6 libdl.so.2 ..

machine2:~/lib$ LD_LIBRARY_PATH=. PYTHONPATH=. python
Python 2.7.3 (default, Feb 27 2014, 19:58:35) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import Foo
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.17' not found (required by ./libstdc++.so.6)
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line 66, in apport_excepthook
    from apport.fileutils import likely_packaged, get_recent_crashes
  File "/usr/lib/python2.7/dist-packages/apport/__init__.py", line 1, in <module>
    from apport.report import Report
  File "/usr/lib/python2.7/dist-packages/apport/report.py", line 20, in <module>
    import apport.fileutils
  File "/usr/lib/python2.7/dist-packages/apport/fileutils.py", line 22, in <module>
    from apport.packaging_impl import impl as packaging
  File "/usr/lib/python2.7/dist-packages/apport/packaging_impl.py", line 20, in <module>
    import apt
  File "/usr/lib/python2.7/dist-packages/apt/__init__.py", line 21, in <module>
    import apt_pkg
ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.17' not found (required by ./libstdc++.so.6)

Original exception was:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.17' not found (required by ./libstdc++.so.6)

Now Python loads, but of course Foo.so depends on a different version of GLIBC than the one installed on the target system:

machine2:~/lib$ /lib/x86_64-linux-gnu/libc.so.6 
GNU C Library (Ubuntu EGLIBC 2.15-0ubuntu10.5) stable release version 2.15, by Roland McGrath et al.
(...)

How can I make this work and successfully import the module Foo in Python on the target system? Assume that I cannot globally install a newer GLIBC version on the target system, nor install GCC 4.9 and compile the C++ libraries locally.

I do have control over the build process on the build system, but for the sake of argument I'd like to assume that I received the shared object files from a third party and cannot modify them either.

The 'python' binary on the target system seems to depend on the older Glibc version. I assumed Glibc to be backwards ABI compatible, ie with a newer version available (as provided by me), everything should work as intended.

Can't really imagine to be the first with this kind of problem, so I assume there must be an easy solution, even if I can't figure it out...

Answer 1

why doesn't providing the newer version of Glibc just work?

This is explained here .

The solution you came up with -- running the loader with --library-path -- is still problematic if you don't have complete glibc installation in . . In particular, tying to access any of the NSS functions ( getgrpnam , getpwnam , etc.) is likely to crash your program because you didn't copy libnss*.so libraries.

"if I used any of these functions, libnss*.so would be listed using ldd Foo.so and thus copied to ., right?"

No, that is wrong. These libraries are dynamically loaded, and not linked to directly.

Is there some way to reliably find out all dependencies?

For a given invocation, you can do this:

LD_DEBUG=libs ./ld-linux-x86-64.so.2 --library-path . `which python` foo.py

This will give you a list of libraries that the particular foo.py requires. That list may change when you run a different script, so this is not a general solution.

And is there some backup for the claim in your linked post that glibc consists of 200+ libraries?

Sure:

dpkg -L libc6 | grep '\.so' | wc -l
300

Some of these are symlinks, but not all. If you want non-symlink count:

for j in $(dpkg -L libc6 | grep '\.so' ); do [[ -L $j ]] || echo $j; done | wc -l
279

Answer 2

Seems the solution is simply to copy the shared library loader from the build system (with all the dependencies present in the current directory) and to invoke Python as:

machine2:~/lib$ ./ld-linux-x86-64.so.2 --library-path . `which python`
Python 2.7.3 (default, Feb 27 2014, 19:58:35) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import Foo
>>> dir(Foo)
[...]