virtualenvwrapper install of Jupyter Notebook uses virtualenv python but global pip & packages

Question

I have a virtualenvwrapper-wrapped virtualenv. Let's call it test. Jupyter is installed through both my global pip and the virtualenv's pip.

$ which python
/usr/bin/python
$ which pip
/usr/local/bin/pip
$ which jupyter
/usr/local/bin/jupyter
$ workon test
(test)$ which python
/home/keon/.virtualenvs/test/bin/python
(test)$ which pip
/home/keon/.virtualenvs/test/bin/pip
(test)$ which jupyter
/home/keon/.virtualenvs/test/bin/jupyter

Now, I have package A installed through /usr/local/bin/pip and package B installed through /home/keon/.virtualenvs/test/bin/pip. (astropy and emcee resp., but I don't believe it matters)

I launch jupyter notebook (in a shell in the test virtualenv). I then run this code in a cell:

import astropy
import emcee

: And get the following error

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-12-43214a4612e2> in <module>()
      1 import astropy
----> 2 import emcee

ImportError: No module named 'emcee'

In a cell, I use the ! shell commands:

!which python
!which pip
!which jupyter
!echo "Pip freeze below:\n---------------------"
!pip freeze

The output is

/home/keon/.virtualenvs/ph20/bin/python
/usr/local/bin/pip
/usr/local/bin/jupyter
Pip freeze below:
---------------------
[snip]
astropy==3.0.3

So my python is the virtualenv one but it seems to be using the global jupyter and pip.

This problem only seems to exist in one of my virtualenvs. If I use a different one, running the same cell described earlier the output is as expected:

/home/keon/.virtualenvs/test2/bin/python
/home/keon/.virtualenvs/test2/bin/pip
/home/keon/.virtualenvs/test2/bin/jupyter

Answer 1

This is a common problem when there are multiple installs of Jupyter Notebook (globally and in a specific virtualenv). As far as what I have experienced, if Jupyter is not installed in the virtualenv, the terminal will start the global install of Jupyter Notebook even if you give the command while a specific virtualenv is activated. Thus, make sure that Jupyter is installed in 'test' environment and you give the 'jupyter notebook' command when the 'test' environment is activated.

In case the above solution does not work and you are sure that the virtualenv's Jupyter instance is running and it is still not able to detect a specific package which is also installed in the same virtualenv, you can try installing the required package from within the code by using the method suggested here: Installing python module within code

If both of these fail, I would suggest you to create a fresh virtualenv, install jupyter in it, install other packages that you require in it and try running the jupyter instance again while the virtualenv is active.

In case you are not comfortable with managing multiple virtualenvs using the virtualenv wrapper and are often getting such errors, especially with jupyter, I would suggest you to use Anaconda and create virtualenvs using the GUI which gives a cleaner management system and makes sure your Jupyter instance uses the right virtualenv.