Importing Python package - “ImportError: No module named…”

Question

I know there are a lot of questions about "ImportError: No module named..." but they ususally seem to boil down to no __init__.py file or the package directory not in $PYTHONPATH . I've checked both of those issues and my issue is not down to them.

I have a project which contains protocol buffer definitions. There's a makefile which generates the source as Python, Java or Go. There's a setup.py file which executes make python . I've run pip install -e . in this directory which generates the source files as expected.

I then have a separate project where I'm trying to use the generated protobufs.

Let me illustrate my projects:

myproject/
├── module
│   ├── __init__.py
│   └── module.py
└── main.py

myprotos/
├── Makefile
├── __init__.py
├── my.proto
├── my_pb2.py (generated by the makefile on install)
├── myprotos.egg-info (generated by setup.py)
│   ├── PKG-INFO
│   ├── SOURCES.txt
│   ├── dependency_links.txt
│   └── top_level.txt
└── setup.py

The source of setup.py is pretty simple:

import subprocess
import sys

from setuptools import setup
from setuptools.command.install import install

class Install(install):
    """Customized setuptools install command - builds protos on install."""
    def run(self):
        protoc_command = ["make", "python"]
        if subprocess.call(protoc_command) != 0:
            sys.exit(-1)
        install.run(self)


setup(
    name='myprotos',
    version='0.0.1',
    description='',
    install_requires=[],
    cmdclass={
        'install': Install,
    }
)

The __init__.py in myprotos simply contains:

import my_pb2

And then the contents of myproject/main.py is:

import sys
sys.path.insert(0, '/path/to/myprotos')

import myprotos

Running this code, python main.py outputs:

Traceback (most recent call last):
  File "main.py", line 12, in <module>
    import myprotos
ImportError: No module named myprotos

What have I missed here? It seems like this should work but I clearly haven't understood something crucial.

Answer 1

Let say you have below structure :

demo_proj
    |
    myproject/
    ├── module
    │   ├── __init__.py
    │   └── module.py
    └── main.py

    myprotos/
    ├── Makefile
    ├── __init__.py
    ├── my.proto
    ├── my_pb2.py
    ├── myprotos.egg-info
    │   ├── PKG-INFO
    │   ├── SOURCES.txt
    │   ├── dependency_links.txt
    │   └── top_level.txt
    └── setup.py

Code in main.py:

import sys
sys.path.insert(0, '/path/to/demo_proj')

import myprotos