How do I debug a built-in Python command, package or module?

Question

I would like to debug some of the basic packages that come with the Python install and/or are built-in packages, including pip and venv .

The desire comes from an error message of file permissions (unable to access a file with an "unprintable file name") some of my team is getting running these commands - see this question for details.

Question

How do you debug the Python source code when trying to catch issues in the main python executable, or when directly running a base python module (see following examples for pip and venv )?

$ python -m pip install --upgrade
$ python -m venv .venv

If it matters, my environment is VSCode, where I am happily able to engage the debugger on any custom script I have written, using the built-in debugger that interacts (I assume) with the main Microsoft Python extension.

Answer 1

You will need to set"justMyCode": false in your launch.json for the debugger to trace into third-party code.

Answer 2

Start by looking at the source code for those modules; the -m switch looks for a package or module to import first. If it's a package, then Python imports the __main__ module in that package and runs it as the main script. If it is a module, the module itself is imported and run as __main__ .

Usually the code is structured such that a function is called you can import directly too. You can then just write a bit of code that imports the same function and calls it the same way the __main__ module would. From there on out it is trivial to run this under a debugger.

Eg pip is a package, so python -m pip will import pip.__main__ and run that as a script. This then triggers:

from pip._internal.cli.main import main as _main  # isort:skip # noqa

if __name__ == '__main__':
    sys.exit(_main())

to be run. You can do the same in VSCode; import pip._internal.cli.main.main and call it.

You can find the source code for these modules by just importing them and printing out the resulting object:

python -c "import pip; print(pip)"

The representation of a module, if loaded from disk, will include it's filename. If the filename ends in /__init__.py it's a package, so you can also double-check that the __main__.py file exists:

python -c "import pip.__main_; print(pip.__main__)"

You can do the same for the venv module. This one is part of the Python standard library, so the documentation actually links directly to the source code , and thevenv.__main__ module just imports venv.main() and calls it.