AWS Elastic Beanstalk Using Python 2 and Python 3 in error?

Question

I have followed the AWS tutorial to deploy a django app using elastic beanstalk ( https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create-deploy-python-django.html ). When I get to the eb environment creation stage using the cli, I encounter errors that indicate Python 2 is somehow in the mix, which shouldn't be the case.

My environment is created, but with errors, as detailed below.

I have considered the following.

-I have re-reviewed the django tutorial and made sure I am following it.
-I have researched the issue on SO and reviewed my eb logs, but I haven't been able to identify insights into this particular problem.
-I have confirmed that pywin 225 exists and the aws platform is configured for python 3.6 (and not 2.7 as the logs suggest?).

The tail end of the env creation log is as follows:

Creating application version archive "app-191001_120338".
Uploading ebdjango/app-191001_120338.zip to S3. This may take a while.
Upload Complete.
Environment details for: ebdjango-dev
  Application name: ebdjango
  Region: us-west-1
  Deployed Version: app-191001_120338
  Environment ID: e-ine5bnjxcv
  Platform: arn:aws:elasticbeanstalk:us-west-1::platform/Python 3.6 running on 64bit Amazon Linux/2.9.2
  Tier: WebServer-Standard-1.0
  CNAME: ebdjango-dev2.us-west-1.elasticbeanstalk.com
  Updated: 2019-10-01 19:03:41.226000+00:00
Printing Status:
2019-10-01 19:03:39    INFO    createEnvironment is starting.
2019-10-01 19:03:41    INFO    Using elasticbeanstalk-us-west-1-307455546112 as Amazon S3 storage bucket for environment data.
2019-10-01 19:04:03    INFO    Created target group named: arn:aws:elasticloadbalancing:us-west-1:307455546112:targetgroup/awseb-AWSEB-BNKO4RVK1UEY/9d7bd9879e64fe8e
2019-10-01 19:04:03    INFO    Created security group named: sg-08eeeeae2daaa824a
2019-10-01 19:04:18    INFO    Created security group named: awseb-e-ine5bnjxcv-stack-AWSEBSecurityGroup-1XHDD3SI17QRF
2019-10-01 19:04:19    INFO    Created Auto Scaling launch configuration named: awseb-e-ine5bnjxcv-stack-AWSEBAutoScalingLaunchConfiguration-1JT4JXF8UL7YO
2019-10-01 19:05:20    INFO    Created Auto Scaling group named: awseb-e-ine5bnjxcv-stack-AWSEBAutoScalingGroup-UOU7OOHIFLPM
2019-10-01 19:05:20    INFO    Waiting for EC2 instances to launch. This may take a few minutes.
2019-10-01 19:05:20    INFO    Created Auto Scaling group policy named: arn:aws:autoscaling:us-west-1:307455546112:scalingPolicy:374ae066-b319-4861-9043-1340fe3c1a1c:autoScalingGroupName/awseb-e-ine5bnjxcv-stack-AWSEBAutoScalingGroup-UOU7OOHIFLPM:policyName/awseb-e-ine5bnjxcv-stack-AWSEBAutoScalingScaleDownPolicy-40CWL7KRUGXQ
2019-10-01 19:05:20    INFO    Created Auto Scaling group policy named: arn:aws:autoscaling:us-west-1:307455546112:scalingPolicy:343a776e-ed77-4af1-a8f0-c0c59c9d3e0b:autoScalingGroupName/awseb-e-ine5bnjxcv-stack-AWSEBAutoScalingGroup-UOU7OOHIFLPM:policyName/awseb-e-ine5bnjxcv-stack-AWSEBAutoScalingScaleUpPolicy-8FST2RMX5LJH
2019-10-01 19:05:20    INFO    Created CloudWatch alarm named: awseb-e-ine5bnjxcv-stack-AWSEBCloudwatchAlarmHigh-1FB67ATMY3PVV
2019-10-01 19:05:20    INFO    Created CloudWatch alarm named: awseb-e-ine5bnjxcv-stack-AWSEBCloudwatchAlarmLow-163VGCKXY40IE
2019-10-01 19:06:10    INFO    Created load balancer named: arn:aws:elasticloadbalancing:us-west-1:307455546112:loadbalancer/app/awseb-AWSEB-15E7TQ3L7T3G4/297b8eee80310810
2019-10-01 19:06:10    INFO    Created Load Balancer listener named: arn:aws:elasticloadbalancing:us-west-1:307455546112:listener/app/awseb-AWSEB-15E7TQ3L7T3G4/297b8eee80310810/b6d5a64f37e36d16
2019-10-01 19:06:22    ERROR   Your requirements.txt is invalid. Snapshot your logs for details.
2019-10-01 19:06:25    ERROR   [Instance: i-0cd6718e8f5bee560] Command failed on instance. Return code: 1 Output: (TRUNCATED)...)
  File "/usr/lib64/python2.7/subprocess.py", line 190, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1.
Hook /opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
2019-10-01 19:06:26    INFO    Command execution completed on all instances. Summary: [Successful: 0, Failed: 1].
2019-10-01 19:07:28    ERROR   Create environment operation is complete, but with errors. For more information, see troubleshooting documentation.

That reference to /usr/lib64/python2.7/subprocess.py is throwing me off, since I am using Python 3.6.1.

When I look into the logs for the 'requirements.txt invalid issue, the error is coming from.

Collecting pywin32==225 (from -r /opt/python/ondeck/app/requirements.txt (line 16))
    Could not find a version that satisfies the requirement pywin32==225 (from -r /opt/python/ondeck/app/requirements.txt (line 16)) (from versions: )
  No matching distribution found for pywin32==225 (from -r /opt/python/ondeck/app/requirements.txt (line 16))

pywin32==225 does indeed exist. I also tried changing requirements.txt version to 223 manually (in a separate attempt) and that did not work. I did this just to try something that was not the latest version of pywin32 to see if I would get a different result.

The error logs continue with another reference to python 2.7.

You are using pip version 9.0.1, however version 19.2.3 is available.
  You should consider upgrading via the 'pip install --upgrade pip' command.
  2019-10-01 19:06:22,598 ERROR    Error installing dependencies: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1
  Traceback (most recent call last):
    File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 22, in main
      install_dependencies()
    File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 18, in install_dependencies
      check_call('%s install -r %s' % (os.path.join(APP_VIRTUAL_ENV, 'bin', 'pip'), requirements_file), shell=True)
    File "/usr/lib64/python2.7/subprocess.py", line 190, in check_call
      raise CalledProcessError(retcode, cmd)
  CalledProcessError: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1 (ElasticBeanstalk::ExternalInvocationError)
caused by: Collecting awsebcli==3.15.3 (from -r /opt/python/ondeck/app/requirements.txt (line 1)) 

I would not expect any python 2 references since this is all configured for python 3.6. Indeed when I log into my AWS console I can confirm the platform is Python 3.6 running on 64bit Amazon Linux/2.9.2.

I would expect to have no issues creating this application/environment using the eb CLI, since I am following the AWS django tutorial exactly.

After receiving these error messages, I started a git repo for the project so that others could review my requirements and config files. https://github.com/rdemint/ebdjango

Thanks in advance

Answer 1

pywin32 is only available on windows . The error log is showing that requirements.txt has not installed packages successfully and has failed on some step hence returning non-zero status.

You can either install use python with wine on your Amazon Linux (you can use container commands to manage the installation packages) or create an EC2 Windows instance, Bake the AMI with required packages, then in the Elasticbeanstalk environment select your custom AMI. Finally, you can deploy your application on Elasticbeanstalk.

Answer 2

I just uninstalled pywin32 and pypiwin32, Use these commands one by one,

pip uninstall pywin32
pip uninstall pypiwin32
pip freeze > requirements.txt

And now push the code to Github after committing. This fixed the issue for me.