Python module not found even when it exists in the path

Question

I am getting a "No module named X" error even when X is clearly present in the system path. The easiest way to explain is with a repro case:

1. Clear a folder and install google-auth-httplib2 locally.

circleci@27190f660cbe:~/project/tempd$ rm -rf *
circleci@27190f660cbe:~/project/tempd$ pip install --target . google-auth-httplib2
Collecting google-auth-httplib2
  Using cached https://files.pythonhosted.org/packages/33/49/c814d6d438b823441552198f096fcd0377fd6c88714dbed34f1d3c8c4389/google_auth_httplib2-0.0.3-py2.py3-none-any.whl
Collecting httplib2>=0.9.1 (from google-auth-httplib2)
Collecting google-auth (from google-auth-httplib2)
  Using cached https://files.pythonhosted.org/packages/c5/9b/ed0516cc1f7609fb0217e3057ff4f0f9f3e3ce79a369c6af4a6c5ca25664/google_auth-1.6.3-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from google-auth->google-auth-httplib2)
  Using cached https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1 (from google-auth->google-auth-httplib2)
  Using cached https://files.pythonhosted.org/packages/da/98/8ddd9fa4d84065926832bcf2255a2b69f1d03330aa4d1c49cc7317ac888e/pyasn1_modules-0.2.4-py2.py3-none-any.whl
Collecting cachetools>=2.0.0 (from google-auth->google-auth-httplib2)
  Using cached https://files.pythonhosted.org/packages/39/2b/d87fc2369242bd743883232c463f28205902b8579cb68dcf5b11eee1652f/cachetools-3.1.0-py2.py3-none-any.whl
Collecting six>=1.9.0 (from google-auth->google-auth-httplib2)
  Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3 (from rsa>=3.1.4->google-auth->google-auth-httplib2)
  Using cached https://files.pythonhosted.org/packages/7b/7c/c9386b82a25115cccf1903441bba3cbadcfae7b678a20167347fa8ded34c/pyasn1-0.4.5-py2.py3-none-any.whl
Installing collected packages: httplib2, pyasn1, rsa, pyasn1-modules, cachetools, six, google-auth, google-auth-httplib2
Successfully installed cachetools-3.1.0 google-auth-1.6.3 google-auth-httplib2-0.0.3 httplib2-0.12.1 pyasn1-0.4.5 pyasn1-modules-0.2.4 rsa-4.0 six-1.12.0
You are using pip version 9.0.1, however version 19.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

2. Verify the module and __init__.py are present.

circleci@27190f660cbe:~/project/tempd$ find google/auth/transport
google/auth/transport
google/auth/transport/grpc.pyc
google/auth/transport/requests.py
google/auth/transport/urllib3.py
google/auth/transport/requests.pyc
google/auth/transport/__init__.py
google/auth/transport/_http_client.pyc
google/auth/transport/urllib3.pyc
google/auth/transport/__init__.pyc
google/auth/transport/grpc.py
google/auth/transport/_http_client.py

3. Attempt to import it in python, but still get error.

circleci@27190f660cbe:~/project/tempd$ python
Python 2.7.13 (default, Sep 14 2017, 23:43:58)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.insert(0, ".")
>>> import google.auth.transport.requests
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named google.auth.transport.requests
>>> exit()

I am seeing this in python 2.7.13. What is going on here, and what is the solution?

Note: I need to maintain support for 2.7.13.

Note2: Unfortunately I am constrained to having to import in this way - virtualenv etc. are not options.

For context, this is a vendoring problem. I am developing a python app that is "installed" by a non-programmer on their system by unzipping a zip file and running python from there. Thus, solutions like virtualenv (or really anything that don't directly solve this in code) aren't going to work.

Answer 1

The problem with your setup is that directory google/ lacks file __init__.py so import google doesn't work. Create google/__init__.py , restart python and try again:

$ touch google/__init__.py
$ python2.7
Python 2.7.13 (default, Sep 26 2018, 18:42:22) 
[GCC 6.3.0 20170516] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.insert(0, ".")
>>> import google.auth.transport.requests