I'm studying how python loads modules.
I'm starting from requests
that loads urllib3
.
I have found those line :
from .packages.six.moves.http_client import (
IncompleteRead as httplib_IncompleteRead
)
in file called src/urllib3/exceptions.py
That import is a relative one, so I'm looking for the folder src/urllib3/packages/six/moves/http_client.py
or src/urllib3/packages/six/moves/http_client/__init__.py
Those files don't exist.
Thankfully src/urllib3/packages/six.py
defines a new module loader, so the import .packages.six.moves.http_client
requires http_client
core python module instead.
But I don't understand how the src/urllib3/packages/six.py
is loaded. I've not found that kind of import.
Is there a way to know which module loads src/urllib3/packages/six.py
directly?
I have tried with __name__
global variable as suggested here . Its value is urllib3.packages.six
but urllib3.packages
doesn't load that file.
The from .packages.six.moves.http_client import ...
expression causes .packages.six
to be loaded first . Python always loads all packages in a nested package reference to a module.
So .packages.six.moves.http_client
causes Python first to look for urllib3.packages
, then for urllib3.packages.six
, and so on. The import machinery does so by looking for the full name in sys.modules
, and if it is not there, triggers a module search and load for each.
The first time this happens, sys.modules['urllib3.packages.six']
doesn't exist yet, the import machinery finds the file urllib3/packages/six.py
, imports that, before it'll look for more names.
And, as you discovered, the very act of importing the six.py
module file, causes that module to add sys.modules['urllib3.packages.six.moves']
and further references to standard library modules.
Python's import machinery is quite a complex beast; the Python reference documentation covers this comprehensively in The import system ; the specific entries to look for are:
A direct call to
__import__()
performs only the module search and, if found, the module creation operation. While certain side-effects may occur, such as the importing of parent packages , and the updating of various caches (including sys.modules), only the import statement performs a name binding operation.
and under Regular packages
Importing
parent.one
will implicitly executeparent/__init__.py
andparent/one/__init__.py.
Subsequent imports ofparent.two
orparent.three
will executeparent/two/__init__.py
andparent/three/__init__.py
respectively.
and under The module cache :
The first place checked during import search is
sys.modules
. This mapping serves as a cache of all modules that have been previously imported, including the intermediate paths . So iffoo.bar.baz
was previously imported,sys.modules
will contain entries forfoo
,foo.bar
, andfoo.bar.baz
. Each key will have as its value the corresponding module object.
(bold emphasis in quoted sections added by me).
Note that everything in the urllib3/packages
directory is a vendorized package; a project that normally would be installed independently, but which the urllib3
project has decided to package up with their own distribution to avoid having to worry about what versions to support. six
is such an independent project, you can install it from PyPI .
You can find more information on the six.moves
virtual package in the six
project documentation . It's purpose is to make it easier for library developers to write code that is compatible both with Python 2 and Python 3, without having to worry about what standard library name to import on either version.
One notable thing about import function in Python is that it can import not only modules themselves but also variables, classes, functions (generally namespaces) from inside them. In the example, you have provided:
from .packages.six.moves.http_client import (
IncompleteRead as httplib_IncompleteRead
)
import statement refers to moves variable from src/urllib3/packages/six.py
module, which is defined at line 316 of that file and being assigned to an instance of class _MovedItems :
moves = _MovedItems(__name__ + ".moves")
and http_client is a property or method of this class instance
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.