当 pip 设置为需要 virtualenv 时，如何在 anaconda3 venv 中允许 pip？

Question

I've just rebuilt my mac environment using the tutorials here:

https://hackercodex.com/guide/mac-development-configuration/ & here: https://hackercodex.com/guide/python-development-environment-on-mac-osx/

I want to require a virtualenv for pip, and have set that by opening:

vim ~/Library/Application\ Support/pip/pip.conf

and adding:

[install]
require-virtualenv = true

[uninstall]
require-virtualenv = true

Then, I followed a guide to set up jupyter notebooks w/tensorflow, because I am trying to follow a udemy course on machine learning that requires both: https://medium.com/@margaretmz/anaconda-jupyter-notebook-tensorflow-and-keras-b91f381405f8

During this tutorial, it mentions that you should use pip install instead of conda install for tensorflow, because the conda package isn't officially supported.

I can install pip on conda just fine by running:

conda install pip

But when I try to run:

pip3 install tensorflow

I get the error:

"Could not find an activated virtualenv (required)."

I know why I'm getting this error, I just don't know how to change my code to ALSO accept use of pip & pip3 inside anaconda venvs.

My anaconda3 folder is inside my Virtualenvs folder, along with all of my other virtual environments.

I've tried temporarily turning off the restriction by defining a new function in ~/.bashrc:

cpip(){
PIP_REQUIRE_VIRTUALENV="0" pip3 "$@"
}

and using that instead, with no luck, not surprisingly.

I think the problem may be here, inside my bash_profile:

# How to Set Up Mac For Dev:
# https://hackercodex.com/guide/mac-development-configuration/
# Ensure user-installed binaries take precedence
export PATH=/usr/local/bin:$PATH
# Load .bashrc if it exists
test -f ~/.bashrc && source ~/.bashrc


# Activate Bash Completion:
if [ -f $(brew --prefix)/etc/bash_completion ]; then
    source $(brew --prefix)/etc/bash_completion
fi


# Toggle for installing global packages:
gpip(){
   PIP_REQUIRE_VIRTUALENV="0" pip3 "$@"
}
# Toggle for installing conda packages:
cpip(){
   PIP_REQUIRE_VIRTUALENV="0" pip3 "$@"
}
# Be sure to run "source ~/.bash_profile after toggle for changes to
take effect.
# Run "gpip install" (i.e. "gpip install --upgrade pip setuptools
wheel virtualenv")


# added by Anaconda3 2018.12 installer
# >>> conda init >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$(CONDA_REPORT_ERRORS=false
'/Users/erikhayton/Virtualenvs/anaconda3/bin/conda' shell.bash hook
2> /dev/null)"
if [ $? -eq 0 ]; then
    \eval "$__conda_setup"
else
    if [ -f
"/Users/erikhayton/Virtualenvs/anaconda3/etc/profile.d/conda.sh" ];
then
        .
"/Users/erikhayton/Virtualenvs/anaconda3/etc/profile.d/conda.sh"
        CONDA_CHANGEPS1=false conda activate base
    else
        \export
PATH="/Users/erikhayton/Virtualenvs/anaconda3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda init <<<

I want to be able to use pip (& pip3, pip2) in both (& only in) anaconda3's activated 'env's and virtualenvs.

Answer 1

When you conda install pip , a new pip is placed inside your anaconda virtualenv's bin/ directory. Each pip knows whether/which virtualenv it's inside of, and each pip only installs packages inside its own virtualenv. You can run it like /Users/erikhayton/Virtualenvs/anaconda3/bin/pip install tenserflow

You can know where pip3 is by running which pip3 .

When you activate a virtualenv, environment variables in your shell are being modified. The virtualenv's bin/ directory is placed in your PATH . If you run /Users/erikhayton/Virtualenvs/anaconda3/bin/activate and then which pip3 , you'll see a different path.

See also Using Pip to install packages to Anaconda Environment

Answer 2

Usually when you use virtual environments, you need to activate them first before you can use them. Somewhere along the line, you would have needed to run a command to create your virtual environment:

virtualenv awesome_virtualenv

Then to make it active:

cd ~/Virtualenvs/awesome_virtualenv
source bin/activate
pip3 install tensorflow  # this will install TensorFlow into your awesome_virtualenv

You can create as many virtual environments as you want and install different sets of libraries in each.

Answer 3

The problem is that pip doesn't recognise the conda environment as being not the global environment. It doesn't look like the pip authors intend to fix this (for good reasons I think btw). On the conda side there seems to be no movement either (considering this github issue that has not seen any movement over the past year). So basically, we'll have to do our own scripting :).

This means that whenever we activate a conda environment, we either need to make it look like we're also in a virtual environment, or we switch off PIP_REQUIRE_VIRTUALENV . The solution below uses the latter option (but I can imagine the former working just as well). There is unfortunately no global activate hook in conda , but there are per environment hooks . So all we need to do is run the following 2 commands in our environment:

    echo "export PIP_REQUIRE_VIRTUALENV=false" > "$CONDA_PREFIX/etc/conda/activate.d/dont-require-venv-for-pip.sh"
    echo "export PIP_REQUIRE_VIRTUALENV=true" > "$CONDA_PREFIX/etc/conda/deactivate.d/require-venv-for-pip.sh"

Now whenever we activate this conda environment, PIP_REQUIRE_VIRTUALENV will be set to false , and it will be reset to true as soon as wel deactivate the environment.

Since we want to (easily) install this is into all our environments, I made a function which I placed in my .zshrc (should work just as well in your .bashrc / bash_profile ).

function allow_pip_in_conda_environment() {
    # abort if we're not in a conda env (or in the base environment)
    if [[ -z "$CONDA_DEFAULT_ENV"  || "$CONDA_DEFAULT_ENV" == "base" ]]; then
        echo "Should be run from within a conda environment (not base)"
        return
    fi
    ACTIVATE="$CONDA_PREFIX/etc/conda/activate.d/dont-require-venv-for-pip.sh"
    DEACTIVATE="$CONDA_PREFIX/etc/conda/deactivate.d/require-venv-for-pip.sh"

    # abort if either the activate or the deactivate hook already exists in this env
    if [[ -f "$ACTIVATE" || -f "$DEACTIVATE" ]]; then
        echo "This hook is already installed in this conda environment"
        return
    fi

    # write the hooks (create dirs if they don't exist)
    mkdir -p "$(dirname "$ACTIVATE")"
    mkdir -p "$(dirname "$DEACTIVATE")"
    echo "export PIP_REQUIRE_VIRTUALENV=false" > "$ACTIVATE"
    echo "export PIP_REQUIRE_VIRTUALENV=true" > "$DEACTIVATE"

    # switch off PIP_REQUIRE_VIRTUALENV in the current session as well
    export PIP_REQUIRE_VIRTUALENV=false
}

Now every time I run into a dreaded Could not find an activated virtualenv (required). , all I need to do is run allow_pip_in_conda_environment , and it fixes it in my current session, and forever after in this conda environment.

(PS: same code also works with mamba)