Python Importing modules in a package

Question

I currently have a module I created that has a number of functions.
It's getting quite large so I figured I should make it into a package and split the functions up to make it more manageable.

I'm just testing out how this all works before I do this for real so apologies if it seems a bit tenuous.
I've created a folder called pack_test and in it I have:
__init__.py
foo.py
bar.py

---

__init__.py contains:

__all__ = ['foo', 'bar']
from . import *
import subprocess
from os import environ

In the console I can write import pack_test as pt and this is fine, no errors.
pt. and two tabs shows me that I can see pt.bar , pt.environ , pt.foo and pt.subprocess in there.
All good so far.

---

If I want to reference subprocess or environ in foo.py or bar.py how do I do it in there?

If in bar.py I have a function which just does return subprocess.call('ls') it errors saying NameError: name 'subprocess' is not defined . There must be something I'm missing which enables me to reference subprocess from the level above? Presumably, once I can get the syntax from that I can also just call environ in a similar way?

The alternative as I could see it would be to have import subprocess in both foo.py and bar.py but then this seems a bit odd to me to have it appear across multiple files when I could have it the once at a higher level, particularly if I went on to have a large number of files rather than just 2 in this example.

Answer 1

TL;DR:

__init__.py :

import foo
import bar

__all__ = ["foo", "bar"]

foo.py :

import subprocess
from os import environ

# your code here

bar.py

import subprocess
from os import environ

# your code here

There must be something I'm missing which enables me to reference subprocess from the level above?

Nope, this is the expected behaviour.

import loads a module (if it isn't already), caches it in sys.modules (idem), and bind the imported names in the current namespace. Each Python module has (or "is") it's own namespace (there's no real "global" namespace). IOW, you have to import what you need in each module, ie if foo.py needs subprocess , it must explicitely import it.

This can seem a bit tedious at first but in the long run it really helps wrt/ maintainability - you just have to read the imports at the top of your module (pep 08: always put all imports at the beginning of the module) to know where a name comes from.

Also you should not use star imports (aka wild card imports aka from xxx import * ) anywhere else than in your python shell (and even then...) - it's a maintainance time bomb. Not only because you don't know where each name comes from, but also because it's a sure way to rebind an already import name. Imagine that your foo module defines function "func". Somewhere you have "from foo import *; from bar import *", then later in the code a call to func. Now someone edits bar.py and adds a (distinct) "func" function, and suddenly you call fails, because you're not calling the expected "func". Now enjoy debugging this... And real-life examples are usually a bit more complex than this.

So if you fancy your mental sanity, don't be lazy, don't try to be smart either, just do the simple obvious thing: explicitely import the names you're interested in at the top of your modules.

(been here, done that etc)

Answer 2

You could create modules.py containing

import subprocess
import os

Then in foo.py or any of your files just have.

from modules import *

Your import statements in your files are then static and just update modules.py when you want to add an additional module accessible to them all.