tox/pytest tests pass/fail depending on whether another module is installed in host environment

Question

I have a number of pytest tests, deployed using tox , that install an old version of a module, then import it, then install a newer version of the same module. The tests check that upgrading a module that has already been loaded raises a warning (to tell the user to restart the session/interpreter).

My problem is that the success of the tests depends on whether the old version of the module is installed in the host environment (eg pip install --user random_module && tox or pip uninstall random_module && tox ). I was very surprised to discover that the tests were sensitive to what I had installed in my user. What is going on, and how do I avoid that interaction so that the tests run the same, regardless of what is installed in my user's site-packages ?

More information about the project

I am writing a wrapper package for pip to allow installing packages from inside a REPL/console/script . One complication that this raises is that someone could have already loaded a module, and then install an upgrade, but the old version will remain in memory. In these cases, I need to warn to user that they need to restart their session for the new installation to take effect.

Answer 1

The issue with two tests switching between failing/passing is the installation of the progress package not being rolled back between test runs. One possible solution would be enforcing recreation of the virtual env by tox via running tox --recreate , as you have suggested in the comments. Another would be implementing a rollback in a fixture, eg

@pytest.fixture
def rollback_progress_package():
    yield  # wait for the test to finish
    subprocess.run(['pip', 'install', '--force-reinstall', 'progress==1.5'])

@pytest.mark.usefixtures('rollback_progress_package')
def test_progress_already_loaded_warning():
    ...

This has the nice effect that the rollback can be done in between the test run and not just once per run when the virtual env is recreated.