如何從Jenkins將Python應用程序部署到Amazon Elastic Beanstalk？

Question

我試圖從Jenkins作業以編程方式部署到Amazon Elastic Beanstalk。 在我的開發機器上，這很簡單：

eb deploy $(AWS_ELASTIC_BEANSTALK_ENVIRONMENT)

在Jenkins上，它應該像將以下配置為構建命令一樣簡單：

virtualenv env && source env/bin/activate && pip install awsebcli
mkdir -p .elasticbeanstalk
cat << EOF > .elasticbeanstalk/config.yml 
branch-defaults:
  master:
    environment:  myenv
global:
  application_name:  myapp
  default_ec2_keyname: null
  default_platform: 64bit Amazon Linux 2014.09 v1.0.9 running Python 2.7
  default_region: us-west-2
  profile: eb-cli
  sc: git
EOF
eb deploy myenv

但是，這會失敗，並帶有以下跟蹤：

[EnvInject] - Loading node environment variables.
Building remotely on standard Amazon Linux 2014.09 AMI (i-d39710df) (x) in workspace /media/ephemeral0/jenkins/workspace/My_Job
Fetching changes from the remote Git repository
Fetching upstream changes from git@github.com:myapp.git
Checking out Revision be45db94111f9ab49fe8031eb583307d2fb9921c (origin/master)
[My_Job] $ /bin/sh -xe /tmp/hudson8633484437962332339.sh
+ virtualenv env
New python executable in env/bin/python2.7
Not overwriting existing python script env/bin/python (you must use env/bin/python2.7)
Installing Setuptools..................................................................................................................done.
Installing Pip....................................................................................................................................done.
+ source env/bin/activate
++ deactivate nondestructive
++ unset pydoc
++ '[' -n '' ']'
++ '[' -n '' ']'
++ '[' -n /bin/sh -o -n '' ']'
++ hash -r
++ '[' -n '' ']'
++ unset VIRTUAL_ENV
++ '[' '!' nondestructive = nondestructive ']'
++ VIRTUAL_ENV=/media/ephemeral0/jenkins/workspace/My_Job/env
++ export VIRTUAL_ENV
++ _OLD_VIRTUAL_PATH=/usr/local/bin:/bin:/usr/bin:/opt/aws/bin
++ PATH=/media/ephemeral0/jenkins/workspace/My_Job/env/bin:/usr/local/bin:/bin:/usr/bin:/opt/aws/bin
++ export PATH
++ '[' -n '' ']'
++ '[' -z '' ']'
++ _OLD_VIRTUAL_PS1=
++ '[' x '!=' x ']'
+++ basename /media/ephemeral0/jenkins/workspace/My_Job/env
++ '[' env = __ ']'
+++ basename /media/ephemeral0/jenkins/workspace/My_Job/env
++ PS1='(env)'
++ export PS1
++ alias 'pydoc=python -m pydoc'
++ '[' -n /bin/sh -o -n '' ']'
++ hash -r
+ pip install awsebcli
Requirement already satisfied (use --upgrade to upgrade): awsebcli in ./env/lib/python2.7/site-packages
Downloading/unpacking setuptools>=7.0 (from awsebcli)
  Running setup.py egg_info for package setuptools

Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.11 in ./env/lib/python2.7/site-packages (from awsebcli)
Requirement already satisfied (use --upgrade to upgrade): six==1.8.0 in ./env/lib/python2.7/site-packages (from awsebcli)
Requirement already satisfied (use --upgrade to upgrade): cement==2.4 in ./env/lib/python2.7/site-packages (from awsebcli)
Requirement already satisfied (use --upgrade to upgrade): python-dateutil>=2.2 in ./env/lib/python2.7/site-packages (from awsebcli)
Requirement already satisfied (use --upgrade to upgrade): jmespath>=0.4.1 in ./env/lib/python2.7/site-packages (from awsebcli)
Installing collected packages: setuptools
  Found existing installation: setuptools 0.9.7
    Uninstalling setuptools:
      Successfully uninstalled setuptools
  Running setup.py install for setuptools

    Installing easy_install script to /media/ephemeral0/jenkins/workspace/My_Job/env/bin
    Installing easy_install-2.7 script to /media/ephemeral0/jenkins/workspace/My_Job/env/bin
Successfully installed setuptools
Cleaning up...
+ mkdir -p .elasticbeanstalk
+ cat
+ cat .elasticbeanstalk/config.yml
branch-defaults:
  master:
    environment: myenv
global:
  application_name: myapp
  default_ec2_keyname: null
  default_platform: 64bit Amazon Linux 2014.09 v1.0.9 running Python 2.7
  default_region: us-west-2
  profile: eb-cli
  sc: git
+ eb deploy myenv
ERROR: The config profile (eb-cli) could not be found
Build step 'Execute shell' marked build as failure
Finished: FAILURE

目前還不清楚為什么會發生這種情況，因為當我在我的項目的本地副本上運行上述內容時，它運行正常。

錯誤消息似乎沒有多大幫助。 目前還不清楚為什么在Jenkins機器上找不到eb-cli。

所以再次總結一下我的問題：如何從Jenkins部署到Amazon Elastic Beanstalk？ 上述方法是否正確，但細節有誤？ 還是完全有一些更簡單的方法？

Answer 1

要config profile (eb-cli) could not be found錯誤，請將用於部署到EB的憑據丟棄到jenkins機器上的jenkins用戶的~/.aws/config中。 如果您在本地計算機上構建了部署，則應該能夠從本地直接從~/.aws/config提取文件。 它看起來像這樣：

[profile eb-cli]
aws_access_key_id = (for your IAM user)
aws_secret_access_key = (for your IAM user)

Answer 2

我通過ssh進入Jenkins機器，運行eb init ，然后將生成的.elasticbeanstalk/config.yml與我上面使用的here-doc中的那個進行比較來解決這個問題。 由於我的開發機器與Jenkins機器的安全配置文件不同，因此兩者不同。

我們可以重寫這個腳本，以便對不同的config.yaml文件更健壯，如下所示：

virtualenv env && source env/bin/activate && pip install awsebcli
echo "1" | eb init myapp --region us-west-2 && eb use myenv && eb deploy myenv

注意，我們使用echo "1" | eb init myapp --region us-west-2 echo "1" | eb init myapp --region us-west-2選擇默認環境，因為eb init不將環境作為位置參數，然后使用eb use myenv來選擇我們想要的環境。

Answer 3

我們突然遇到了這個問題，這可能或者可能不相關，但是我想讓那些可能再遇到這個問題的人知道這個問題。 eb-cli似乎略有改變，不允許全局設置憑據。

我們的JenkinsFile看起來像這樣

 withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', 
 credentialsId: 'iam-creds']]) {
       sh "pip install awsebcli --upgrade --user"
       sh "~/.local/bin/eb use my-application"
       sh "~/.local/bin/eb deploy --verbose"
     }

我們的config.yml看起來像這樣

 global:
   application_name: my-application
   default_ec2_keyname: application-keyname
   ...
   profile: eb-cli
   sc: git
   workspace_type: Application

刪除profile密鑰解決了問題...但是，這不允許從本地計算機部署（除非有辦法使用全局變量）