AWS errors when installing requirements with python 2.7 when EB environment is python 3.6

Question

How do I get the aWS EB instance to use the python 3 version that is already installed on the instance? I can't get a new environment running with Python 3.6 running Django 2.1+.

Local (not in virtual env):

which python -> /usr/local/bin/python
python -V -> Python 2.7.15
which python3 -> /usr/local/bin/python3
python3 -V -> Python 3.6.5
which pip -> /usr/local/bin/pip
pip -V -> pip 18.0 from /usr/local/lib/python3.6/site-packages/pip (python 3.6)`
which pip3 -> /usr/local/bin/pip3
pip3 -V -> pip 18.0 from /usr/local/lib/python3.6/site-packages/pip (python 3.6)

Local in virtualenv mstFirst :

which python -> /Users/me/.virtualenvs/mstFirst/bin/python
which python3 -> /Users/me/.virtualenvs/mstFirst/bin/python
python -V -> Python 3.6.5
python3 -V -> Python 3.6.5
which pip -> /Users/me/.virtualenvs/mstFirst/bin/pip
which pip3 -> /Users/me/.virtualenvs/mstFirst/bin/pip3
pip -V -> pip 18.0 from /Users/me/.virtualenvs/mstFirst/lib/python3.6/site-packages/pip (python 3.6)
pip3 -V -> pip 18.0 from /Users/me/.virtualenvs/mstFirst/lib/python3.6/site-packages/pip (python 3.6)
aws --version -> aws-cli/1.16.9 Python/3.6.5 Darwin/17.7.0 botocore/1.11.9
eb --version -> EB CLI 3.14.4 (Python 3.6.5)
eb -list -> * mst-p3
eb -config ->
  ApplicationName: mst
  DateUpdated: 2018-09-07 02:15:51+00:00
  EnvironmentName: mst-p3
  PlatformArn: arn:aws:elasticbeanstalk:us-east-1::platform/Python 3.6 running on 64bit Amazon Linux/2.7.3
  settings: …

Files :

".elasticbeanstalk/config.yml"

branch-defaults:
  master:
    environment: mst-p3
environment-defaults:
  mst-p3: …
global:
  application_name: mst
  default_platform: 64bit Amazon Linux 2018.03 v2.7.3 running Python 3.6
  default_region: us-east-1
  workspace_type: Application
…

".ebextensions/01_packages.config"

packages:
  yum:
    git: []
    python36: []
    python36-devel: []
    postgresql93: []
    postgresql93-devel: []
    mod24_ssl : []

commands:
  pip_upgrade:
    command: /opt/python/run/venv/bin/pip install --upgrade pip
    ignoreErrors: false

Error when trying eb deploy , which points to wrong python version (2.7) while trying to run pip install -r requirements.py :

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-jt3_s4in/slugger/
2018-09-07 03:28:35,315 ERROR    Error installing dependencies: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1
Traceback (most recent call last):
  File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 22, in main
    install_dependencies()
  File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 18, in install_dependencies
    check_call('%s install -r %s' % (os.path.join(APP_VIRTUAL_ENV, 'bin', 'pip'), requirements_file), shell=True)
  File "/usr/lib64/python2.7/subprocess.py", line 186, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1 (Executor::NonZeroExitStatus)

Remote Instance ssh ( eb ssh ) :

pyhthon -V -> Python 2.7.14
pyhthon3 -V -> Python 3.6.5

Things I've tried:

• Replacing the environment AMI with a current version and rebuilding per https://stackoverflow.com/a/47157777/1234721 , but the environment errors with an AMI of "ami-0ff8a91507f77f867" for "Amazon Linux AMI 2018.03.0 (HVM), SSD Volume Type"
• trying to hook into the YAML config to switch the python versions
• re-init using eb init
• verified my ssh keys per: https://stackoverflow.com/a/51669982/1234721

Answer 1

Using eb init definitely works:

eb init -p "64bit Amazon Linux 2017.09 v2.7.3 running Python 3.6" ...

Then just remove all the python stuff from your .ebextension files.

For the yum section , all I use is:

packages:
  yum:
    postgresql94-devel: []

There should be no need to upgrade pip.

Answer 2

Your problem is caused by the wrong version of pip (Python 3.x pip) in the Python 2.7 installation.

The AWS tools are written for Python 2.7

I am also not sure if aws cli or eb has been completely tested with pip 18.

Uninstall pip from the Python 2 directory. Uninstall eb

Reinstall.

I recommend installing the AWS CLI first even if you are not using it. This will ensure that your Python environment is setup correct for AWS tools.

You did not specify the OS, so I cannot give you OS specific uninstall / install directions.

Answer 3

Try:

• backing up and delete.elasticbeanstalk/config.yml
• run: eb init You can then select python version 3.6 within eb init