Python 3.6 import at runtime

Question

I'm currently writing a program where Python-Files are copied into a docker-container with a main module and should then be executed in Threads.

Suppose I have the following structure:

.
├── app.py
├── rule1
|   └── foo.py
├── rule2
|   └── bar.py
:
:   ...
: 
└── ruleX
    └── foobar.py

The idea is, that each .py in a subdirectory implements an interface and is then called in it's own Thread by the main.py . The main.py on the other hand then communicates with the program running on the host. The only thing the Thread would have to to, would be to create an instance of the loaded class and call the .analyze() -function.

As far I found the following code:

import importlib.util
spec = importlib.util.spec_from_file_location("module.name", "/path/to/file.py")
foo = importlib.util.module_from_spec(spec)
spec.loader.exec_module(foo)
foo.MyClass()

But in my case I am not sure what to enter for module.name . And how can the function be called?

Answer 1

The place to look for this information is the documentation . The code that you apparently found somewhere is basically just one of the examples in the docs with all the explanation and context stripped out.

You do also need to understand a little bit about how Python modules and packages work, but otherwise, anything you don't understand here should be findable within the same doc chapter.

---

But in my case I am not sure what to enter for module.name .

module.name is just the name of the module. It's the name that will show up in tracebacks, and it may affect other debugging, and reflective code (eg, if you're using inspect anywhere), and possibly relative imports from within the module. (I'm guessing that, other that debugging output, none of this is going to affect you, in which case it doesn't even matter that much if you get it wrong.)

Anyway, if you're not sure, and want to see some examples rather than dig through dry and confusing documentation, open the interactive interpreter and try this:

import <whatever>
print(<whatever>.__name__, <whatever>.__file__)

… for different values of <whatever> (from the stdlib, and from packages you've installed off PyPI, and from your own project) to get an idea. The __name__ you see there is the name that got passed to spec_from_file_location by the machinery under the covers.

But the short version is:

• If you want rule1 to act like a "namespace package", use rule1.foo .
• If you want rule1 to act like just another path on sys.path, use foo .

---

And how can the function be called?

The exact same way you were calling a class:

foo.analyze()

---

One last thing: what you're building is a pretty standard plugin system, with the simplest possible plugin API ("must expose a function analyze that takes no parameters"). You might want to look around for other applications with plugin systems to read their code for ideas. (But be aware that many of them are made to be backward compatible with Python 2.7 or 3.3, which means that the actual steps for importing will be very different, and you don't want to use the old-style code they use.)