AWS Pull latest code from codecommit on EC2 Instance startup

Question

There seem to be lot of discussion around this topic however nothing precisely for my situation and hasn't resolved it for me so far.

I have my code placed in aws codecommit.

I have created an AMI for one of my running Ubuntu instance in AWS and created a launch configuration using this AMI along with an auto scaling group.

I want to base/modify my launch config AMI every month or so to ensure the AMI itself has recent updated code and so newly launched instances (thru auto scaling) can just pull latest changes from codecommit repo on launch - resulting in reduced launch time.

To achieve this, I placed below code in User data (cloud-init) script and selected a IAM role that has full permissions over all EC2 and codecommit as well as IAM:Passrole permission. However on launch, the script always throws error and does not pull changes (I intentionally kept a file in repo to test)

Option 1

#!/bin/bash
git config --global credential.helper '!aws codecommit credential-helper $@'
git config --global credential.UseHttpPath true
cd /path/to/my/folder/
git remote set-url origin https://git-codecommit.ap-southeast-2.amazonaws.com/v1/repos/reponame
git pull origin master

It throws below error

Error
fatal: $HOME not set
fatal: $HOME not set
fatal: Not a git repository (or any of the parent directories): .git
fatal: could not read Username for 'https://git-codecommit.ap-southeast-2.amazonaws.com': No such device or address

Option 2 -

Tried this option as well with SSH (although haven't tried any further fixes for this)

#!/bin/bash
git config --global credential.helper '!aws codecommit credential-helper $@'
git config --global credential.UseHttpPath true
cd /path/to/my/folder/
git remote set-url origin ssh://git-codecommit.ap-southeast-2.amazonaws.com/v1/repos/reponame
git pull origin master

Got a different error -

Errpr: 
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.

Can someone please hep me understand where I am going wrong?

Thanks.

Answer 1

In Option 1, it looks like the home directory wasn't created yet. When you are setting the global git config, it will go into the home directory's .gitconfig file. Though the option doesn't need to be global, eg you can switch the order of the lines to:

cd /path/to/my/folder/ git config credential.helper '!aws codecommit credential-helper $@' git config credential.UseHttpPath true

This is provided that you have set up EC2 instance roles correctly and that your AWS CLI is able to get the EC2 instance role credentials from EC2 metadata to call AWS APIs.

Though its unclear from the output whether the AWS CLI is installed. The CLI needs to be installed for the git config lines you've posted to work because its going to call "aws codecommit credential-helper" to get a temporary username and password based on the instance role credentials.

In Option 2, you do not need to use the credential helper at all. I am sorry if that was not clear in the documentation. You do, however, need to upload a public key to IAM (instructions here: http://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-ssh-unixes.html#setting-up-ssh-unixes-keys )

You will also need to figure out a way to distribute your public and private key pair to the EC2 instances that you are trying to scale up, this can be quite troublesome.

You can also generate static credentials for CodeCommit ( http://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-gc.html#setting-up-gc-iam ) and put them on your EC2 instance in something like a .netrc file.

IMO Option 1 seems the most secure since you don't have to deal with passing secrets around.