AWS Elastic Beanstalk using PHP with Private Composer Repositories

Question

How do I utilize private composer repositories when deploying with Amazon AWS Elastic Beanstalk in a PHP environment? Specifically using GitHub (Q & A style, answer following)

Answer 1

We needed to use a private library for one of our PHP projects we were deploying via AWS's Elastic Beanstalk (EB). This private library is hosted on GitHub, although similar git hosting (your own server, BitBucket, etc.) probably has similar authentication and could use this solution for deployment.

We used SSH credentials to get at the private git repository. Since we are using GitHub, we used GitHub's Deploy Keys ( https://help.github.com/articles/managing-deploy-keys#deploy-keys ) These keys allow read only access to a specific repository, which is perfect for our needs. Evaluate the best solution for your needs, GitHub has great pros and cons listed for each method.

Our chosen solution embeds the deploy key in with the repository. This is a bit of a security hole. We are dealing with all private repos, with (ideally) secure servers, but this still is a bit of a security risk.

All of this ended up being a bit of a hassle with the way the PHP stack is deployed with Elastic Beanstalk, composer.json was getting auto-run too early and the keys weren't in place beforehand. We found a workaround.

This assumes you already have your deployment setup, but are just stuck at deploying keys. We used the eb cli tools provided by AWS (eb init, eb branch, eb start, etc.) to get things up and going, as well as the git hooks, git aws.push to deploy.

Once we have our Deploy Keys, we can add our library to our composer.json file using the SSH address:

{
...
"require": {
        "repository/project": ">=1.0.0"
},
...
"repositories": [
    {
        "type": "git",
        "url":  "git@github.com:repository/project.git"
    }
]
}

Configure your .gitignore so the composer.lock file is committed and in your repository as well as the vendor folder without it's contents:

[remove composer.lock from file if it exists]
vendor/*

We prefer keeping the composer.lock file in the repository anyway as it locks in the version used in testing. When we move to a production environment we ensure the application is running with the same libraries we tested against. The vendor folder is required to trick EB into not auto-running the composer.phar install process. We need it to wait until we have the ssh keys in place.

Setting up the keys: I couldn't find a good way to affiliate the key and accept github.com as a known_host via scripting. I ended up SSHing to the EB managed server with the software half deployed, added the id_rsa and id_rsa.pub key files to the ~root/.ssh/ (with 400 perms remember!) then trying ssh -T git@github.com (as github recommends) This will prompt to accept the host and add an entry to the ~root/.ssh/known_hosts file. Copy the contents of this file to where you are working on the project.

We are creating all of the setup scripts in the .ebextensions/ folder to configure the Linux server for deployment. This folder is removed (from what I can tell) from the server after pre deployment stage. We are using the PHP 5.5 64bit Amazon AMI solution. Move the id_rsa and id_rsa.pub keys into the the new .ebextensions folder. Also add a file called known_hosts to the folder with the known_hosts contents we provided earlier. Now that we have the 3 files we need, we need to create a final deployment instruction file : 01-github-deploy-keys.config (name the file however you like)

container_commands:
    11-move-priv-key:
        command: "mv ~root/.ssh/id_rsa ~root/.ssh/id_rsa.bak; cp .ebextensions/id_rsa ~root/.ssh/id_rsa; chmod 400 ~root/.ssh/id_rsa;"
    12-move-pub-key:
        command: "mv ~root/.ssh/id_rsa.pub ~root/.ssh/id_rsa.pub.bak; cp .ebextensions/id_rsa.pub ~root/.ssh/id_rsa.pub; chmod 400 ~root/.ssh/id_rsa.pub;"
    12-known-hosts:
        command: "mv ~root/.ssh/known_hosts ~root/.ssh/known_hosts.bak; cp .ebextensions/known_hosts ~root/.ssh/known_hosts; chmod 644 ~root/.ssh/known_hosts;"
    20-install-composer:
        command: "./composer.phar install;"

Remember YAML files uses 4 spaces, not tabs! See the AWS documentation for how these container_commands work: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html#customize-containers-format-commands They will run after the files are pulled from the repository. These commands in "container_commands" section have a working directory of your project, so local paths are preferred.

Add all of these files need to be added and committed to the repository. Run your git aws.push to deploy.

In order to test the setup properly you will need to remove the server from the EB solution stack and re-add it. I just go into the EC2 control panel and find the managed server for this project and terminate it. EB will automatically create a new one for you and attach it once it is ready. Double check your logs, specifically the /var/log/cfn-init.log section. It is probably best to turn off SSH access to the servers via security group at this point. I believe EB restricts logins to root over SSH but just to be sure you may want to disable SSH access all together via firewall/security groups. You shouldn't need to ssh into individual boxes for configuration as they should be seen as volatile.

This was written as a Q & A on 2014-02-20, please post any comments or fixes.

Thanks, - Seth

Answer 2

TLDR: Use ~/.composer/auth.json, github-oauth on composer.json, or create a custom script like the one below:

---

This is my 02-github-deploy-keys.config file. It's working right now. The only workaround was to disable StrictHostKeyChecking. But you can turn StrictHostKeyChecking on after this script runs, if you like.

I added /vendor (without any file) to Git to stop AWS from auto-running Composer before the keys were OK. To do so i created a .gitignore file inside /vendor, with this:

*
!.gitignore

I'm storing the keys (id_rsa) on a S3 bucket, where i allowed "Authorized" people to read the file, but you can put the file on your github repository. These keys were generated on a machine user ( https://developer.github.com/guides/managing-deploy-keys/#machine-users ).

files:
    "/home/ec2-user/sshgit/composer.sh":
        mode: "00755"
        owner: ec2-user
        group: ec2-user
        encoding: plain
        content: |
            if [ ! -f /home/ec2-user/id_rsa ] ; then
              aws s3 cp s3://eb-files/id_rsa /home/ec2-user/id_rsa
              chmod 0400 /home/ec2-user/id_rsa
            fi

            eval `ssh-agent -s`
            ssh-add /home/ec2-user/id_rsa

            echo 'StrictHostKeyChecking no' >> /etc/ssh/ssh_config

            export COMPOSER_HOME=/root
            COMPOSER_HOME=/root
            /opt/elasticbeanstalk/support/composer.phar install --no-interaction

container_commands:
    01-run-composer:
        command: "/home/ec2-user/sshgit/composer.sh"

Just wanted to point out that there's an easier (maybe riskier) way to do this by adding this to composer.json:

"config": {
   "github-oauth": {
        "github.com": "YOUR-OAUTH-KEY"
    }
}

And there's a 3rd way which i did't test, but you can create a ~/.composer/auth.json, and composer will probably understand your tokens there.

Answer 3

I struggled with this. I've got repos in AWS CodeCommit and was looking for the path of least resistance to solve it. I tried ~/.composer/auth.json but it appears that composer is run before I could get the file in place etc. etc.

I went for an approach which includes the vendor directory in my repo (get rid of .git folders within so that it doesn't treate them as submodules) and the whole thing then gets published to Elastic Beanstalk via zip file including that folder.