Using a python 3.6 venv for python 3 modules when python 2.7 and 3.6 are both installed

Question

I think I don't get the venv module in Python 3. I have an AWS EC2 instance in which with yum there is installed (for python) what I listed below… and so the default python and pip is for 2.7 while the python3 command is 3.6.5. Listing pip freeze there's a bunch of stuff installed by default for 2.7, but running python3 -m pip freeze shows me nothing seems to be installed for 3.6.5. I thought I could install some modules more safely by setting up a venv for my work with a python 3 script.

Using python3 -m venv ~/venv/aws I was able to then . venv/aws/bin/activate . venv/aws/bin/activate no problem. Now python --version is still 2.7.14, but I figured a good test for installing something for python3 in the activated venv would be python3 -m pip install --upgrade pip . This doesn't work, because first it says it's uninstalling pip 9.0.3, and it moves on to and error in linking:

Collecting pip
  Using cached https://files.pythonhosted.org/packages/c2/d7/90f34cb0d83a6c5631cf71dfe64cc1054598c843a92b400e55675cc2ac37/pip-18.1-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 9.0.3
    Uninstalling pip-9.0.3:
Exception:
Traceback (most recent call last):
  File "/usr/lib64/python3.6/shutil.py", line 544, in move
    os.rename(src, real_dst)
OSError: [Errno 18] Invalid cross-device link: '/usr/bin/pip' -> '/tmp/pip-183iq_uq-uninstall/usr/bin/pip'

The issue I believe is that when activating the venv I am not getting the venv/aws/bin used for pip pip3 python nor python3 and so… what's wrong with the activate, was I supposed to create this differently?

The list of yum packages:

libxml2-python27.x86_64              2.9.1-6.3.52.amzn1            @amzn-updates
newt-python27.x86_64                 0.52.11-3.11.amzn1            installed
python26.x86_64                      2.6.9-2.89.amzn1              @amzn-main
python26-libs.x86_64                 2.6.9-2.89.amzn1              @amzn-main
python27.x86_64                      2.7.14-1.123.amzn1            @amzn-updates
python27-PyYAML.x86_64               3.10-3.10.amzn1               installed
python27-babel.noarch                0.9.4-5.1.8.amzn1             installed
python27-backports.x86_64            1.0-3.14.amzn1                installed
python27-backports-ssl_match_hostname.noarch
                                     3.4.0.2-1.12.amzn1            installed
python27-beautifulsoup4.noarch       4.6.0-1                       @emr-platform
python27-boto.noarch                 2.48.0-1.2.amzn1              installed
python27-botocore.noarch             1.8.13-1.66.amzn1             @amzn-main
python27-chardet.noarch              2.0.1-7.7.amzn1               installed
python27-colorama.noarch             0.2.5-1.7.amzn1               installed
python27-configobj.noarch            4.7.2-7.15.amzn1              installed
python27-crypto.x86_64               2.6.1-1.15.amzn1              @amzn-main
python27-daemon.noarch               1.5.2-1.5.amzn1               installed
python27-dateutil.noarch             2.1-1.3.amzn1                 installed
python27-devel.x86_64                2.7.14-1.123.amzn1            @amzn-updates
python27-docs.noarch                 2.7.10-1.21.amzn1             @amzn-main
python27-docutils.noarch             0.11-1.15.amzn1               installed
python27-ecdsa.noarch                0.11-3.3.amzn1                installed
python27-futures.noarch              3.0.3-1.3.amzn1               installed
python27-imaging.x86_64              1.1.6-19.9.amzn1              installed
python27-iniparse.noarch             0.3.1-2.1.9.amzn1             installed
python27-jinja2.noarch               2.7.2-2.15.amzn1              installed
python27-jmespath.noarch             0.9.2-1.12.amzn1              installed
python27-jsonpatch.noarch            1.2-2.5.amzn1                 installed
python27-jsonpointer.noarch          1.0-3.4.amzn1                 installed
python27-kitchen.noarch              1.1.1-5.6.amzn1               installed
python27-libs.x86_64                 2.7.14-1.123.amzn1            @amzn-updates
python27-lockfile.noarch             0.8-3.5.amzn1                 installed
python27-markupsafe.x86_64           0.11-4.6.amzn1                installed
python27-nltk.noarch                 3.3-1                         @emr-platform
python27-nose.noarch                 1.3.0-1.11.amzn1              @amzn-main
python27-numpy.x86_64                1.14.5-1                      @emr-bigtop
python27-paramiko.noarch             1.15.1-2.6.amzn1              @amzn-main
python27-pip.noarch                  9.0.3-1.26.amzn1              @amzn-updates
python27-ply.noarch                  3.4-3.12.amzn1                installed
python27-pyasn1.noarch               0.1.7-2.9.amzn1               installed
python27-pycurl.x86_64               7.19.0-17.12.amzn1            installed
python27-pygpgme.x86_64              0.3-9.12.amzn1                installed
python27-pyliblzma.x86_64            0.5.3-11.6.amzn1              installed
python27-pystache.noarch             0.5.3-2.8.amzn1               installed
python27-pyxattr.x86_64              0.5.0-1.6.amzn1               installed
python27-requests.noarch             1.2.3-5.10.amzn1              installed
python27-rsa.noarch                  3.4.1-1.8.amzn1               installed
python27-sagemaker_pyspark.noarch    1.1.0-1                       @emr-bigtop
python27-setuptools.noarch           36.2.7-1.33.amzn1             installed
python27-simplejson.x86_64           3.6.5-1.12.amzn1              installed
python27-six.noarch                  1.8.0-1.23.amzn1              installed
python27-test.x86_64                 2.7.14-1.123.amzn1            @amzn-updates
python27-tools.x86_64                2.7.14-1.123.amzn1            @amzn-updates
python27-urlgrabber.noarch           3.10-8.16.amzn1               installed
python27-urllib3.noarch              1.8.2-1.5.amzn1               installed
python27-virtualenv.noarch           15.1.0-1.14.amzn1             installed
python27-windmill.noarch             1.6-1                         @emr-platform
python34.x86_64                      3.4.8-1.39.amzn1              @amzn-updates
python34-beautifulsoup4.noarch       4.6.0-1                       @emr-platform
python34-boto.noarch                 2.48.0-1                      @emr-platform
python34-dateutil.noarch             2.2-1                         @emr-platform
python34-jmespath.noarch             0.9.3-1                       @emr-platform
python34-libs.x86_64                 3.4.8-1.39.amzn1              @amzn-updates
python34-lxml.x86_64                 4.2.1-1                       @emr-platform
python34-mysqlclient.x86_64          1.3.12-1                      @emr-platform
python34-nltk.noarch                 3.3-1                         @emr-platform
python34-nose.noarch                 1.3.7-1                       @emr-platform
python34-numpy.x86_64                1.14.5-1                      @emr-bigtop
python34-pytz.noarch                 2018.4-1                      @emr-platform
python34-pyyaml.x86_64               3.12-1                        @emr-platform
python34-sagemaker_pyspark.noarch    1.1.0-1                       @emr-bigtop
python34-six.noarch                  1.11.0-1                      @emr-platform
python34-windmill.noarch             1.6-1                         @emr-platform
python36.x86_64                      3.6.5-1.9.amzn1               @amzn-updates
rpm-python26.x86_64                  4.11.3-21.75.amzn1            amzn-main
rrdtool-python26.x86_64              1.4.8-3.18.amzn1              amzn-main
rrdtool-python27.x86_64              1.4.8-3.18.amzn1              amzn-main
samba-python.x86_64                  4.6.2-12.37.amzn1             amzn-main
setools-libs-python.x86_64           3.3.7-34.23.amzn1             amzn-main
subversion-python26.x86_64           1.9.7-1.58.amzn1              amzn-main
subversion-python27.x86_64           1.9.7-1.58.amzn1              amzn-main
systemtap-runtime-python2.x86_64     3.2-2.35.amzn1                amzn-updates
unbound-python.x86_64                1.6.6-1.4.amzn1               amzn-updates
yum-metadata-parser-python26.x86_64  1.1.4-10.20.amzn1             amzn-main
yum-python26.noarch                  3.4.3-150.70.amzn1            amzn-main

Answer 1

Use virtualenv to create the environment

if it is not already installed do so by

• yum install virtualenv

and then set with -p the path to the python3 interpreter you wish to use

• virtualenv -p /path/to/python3 /path/to/virtual/environment