import from a module imported in __init__.py

Question

I previously had had the following structure (as part of a bigger project):

lib/
  __init__.py
  somemodule.py

Where __init__.py was empty and somemodule.py included some method def hello() . Now all across my code repository there were numerous imports of that method, specifically from lib.somemodule import hello .

For various reasons, I now want somemodule.py to be moved into a subpackage. Meaning, I want the structure to be:

lib/
  sublib/
    __init__.py
    somemodule.py
  __init__.py

And figured that if now lib/__init__.py would contain: from sublib import somemodule , running from lib.somemodule import hello would still work (in essence, making the change of structure "transparent" for now).

However, if some external code runs from lib.somemodule import hello , the error ImportError: No module named somemodule is raised, whilst running from lib import somemodule works.

I'd appreciate an explanation on what I'm doing wrong, along with any suggestion on how to make from lib.somemodule import hello run despite the folder structure change.

Answer 1

If you want from lib.somemodule import hello to work, you need to have a module named somemodule in your package lib . Calling from sublib import somemodule will only add your module to the scope of your package, not define a new module, which the import mechanism can find. That's also why from lib import somemodule works. It only imports the global variable somemodule from the package lib .

So having a module lib/somemodule.py with contents

from .sublib.somemodule import *

would be the simplest and cleanest way to resolve that issue, maybe including some kind of DeprecationWarning . Then remove it some time in the future.

Can do

The following ideas are based on informations found in the documentation for the import statement .

The first place checked is sys.modules , the cache of all modules that have been imported previously.

So a simple can do solution is writing the module cache yourself:

import sys
from .sublib import somemodule
sys.modules[__package__+'.somemodule'] = somemodule

Maybe adding a finder to sys.meta_path might be a little cleaner:

import sys
from .sublib import somemodule
class SomemoduleFinder(object):
    def find_module(self, fullname, path=None):
        if fullname == __package__+'.somemodule':
            return self
    def load_module(self, fullname):
        return somemodule
sys.meta_path.append(SomemoduleFinder())

If you had a setup before, where lib/__init__.py didn't import somemodule directly, you could also place the import inside of the SomemoduleFinder.load_module method to mirror that behaviour. Though this only worked with Python 2 for me.

Additionally

Have a look at Python Module Imports - Explicit vs Implicit Relative Imports to prevent portability issues in the future when porting to Python 3.