Python using built-in module instead of custom package found in sys.path

Question

I am having an odd issue with Python for an application I support. The application was working until recently, but now one of its services is failing to start with a Python import error. No changes have been made that I can attribute to this problem.

The application provides its own site-packages directory that contains a backports package, with a configparser package inside. I am not the dev for this, it is commercial software that was in a working config until recently, so I think this is more of a systems issue than a development one.

The error is:

    Traceback (most recent call last):
      File "<app_dir>/tools/vault/bin/vault-unseal.py", line 12, in <module>
        import configparser
      File "<app_dir>/python-site/lib/python2.7/site-packages/configparser.py", line 11, in <module>
        from backports.configparser import (
    ImportError: No module named configparser

My sys.path is:

['',
 '/usr/lib/python2.7/site-packages/pip-19.3.1-py2.7.egg',
 '<app_dir>/python-site/lib/python2.7/site-packages',
 '<app_dir>/python-site/lib64/python2.7/site-packages',
 '/usr/lib64/python27.zip',
 '/usr/lib64/python2.7',
 '/usr/lib64/python2.7/plat-linux2',
 '/usr/lib64/python2.7/lib-tk',
 '/usr/lib64/python2.7/lib-old',
 '/usr/lib64/python2.7/lib-dynload',
 '/usr/lib64/python2.7/site-packages',
 '/usr/lib64/python2.7/site-packages/gtk-2.0',
 '/usr/lib/python2.7/site-packages']

Both these package directories exists with an __init__.py :

<app_dir>/python-site/lib/python2.7/site-packages/backports
<app_dir>/python-site/lib/python2.7/site-packages/backports/configparser

If I start up a python shell and run 'import backports' it will succeed, but instead of importing the correct custom version ( <app_dir>/python-site/lib/python2.7/site-packages/backports ) it will instead import a built-in. That built-in lacks the configparser package. Why is this happening? The correct directory with the custom package is listed higher in the sys.path, so it should import that one instead, right?

>>> import backports
>>> print(backports.__path__)
['/usr/lib/python2.7/site-packages/backports']
>>> backports
<module 'backports' (built-in)>

I was referring to this thread Python ImportError when module is in sys.path which has a similar issue. I believe that none of the 3 listed solutions apply though. PYTHONPATH does contain the package directory; the packages are structured correctly with __init__.py ; and only one version of python is in use on the system (as far as I can tell; if there is a simple way to validate this I would do it).

I'm at a loss, any ideas would be very appreciated.

Answer 1

So I just had basically the same problem, though it was my system python that got into a rather unfortunate state. You need to figure out where it is finding backports/__init__.py and make sure you have configparser installed into that folder. In my case, it appeared I had a backports folder in both /usr/lib/python2.7/site-packages and /usr/lib64/python2.7/site-packages . The later had configparser in it, the former did not. The former is where it was actually loading the __init__.py from. I figured this out by doing

from backports import functools_lru_cache

and then printing out the path where it found functools_lru_cache . If you don't have that module installed (or some other backported module), you're just going to have to try trial and error. You could try to put print statements into all the __init__.py files in all the backport folders on your machine and see which one triggers.

Once I installed configparser into the backports folder in the location where it was being loaded from, I was able to load everything fine.

> python
Python 2.7.18 (default, Apr 23 2020, 09:27:04) [GCC] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import backports
>>> print backports
<module 'backports' (built-in)>
>>> from backports import configparser
>>> print configparser
<module 'backports.configparser' from '/usr/lib/python2.7/site-packages/backports/configparser/__init__.pyc'>