AWS Codebuild terraform provider

Question

Running terraform deploy in codebuild with the following buildspec.yml. Seems terraform isn't picking up the IAM permissions provided by the codebuild role. We're using terraform's remote state (state file is stored in s3), when terraform attempts to contact the S3 bucket containing the state file it dies asking for the terraform provider to be configured:

Downloading modules (if any)...
Get: file:///tmp/src486521661/src/common/byu-aws-accounts-tf
Get: file:///tmp/src486521661/src/common/base-aws-account-
...
Error configuring the backend "s3": No valid credential sources found for AWS Provider.

Here's the buildspec.yml:

version: 0.1
phases:
  install:
    commands:
      - cd common && git clone https://eric.w.nord@gitlab.com/aws-account-tools/acs.git
      - export TerraformVersion=0.9.3 && cd /tmp && curl -o terraform.zip https://releases.hashicorp.com/terraform/${TerraformVersion}/terraform_${TerraformVersion}_linux_amd64.zip && unzip terraform.zip && mv terraform /usr/bin
  build:
    commands:
      - cd accounts/00/dev-stack-oit-byu && terraform init && terraform plan && echo terraform apply

Answer 1

EDIT: THE BUG HAS BEEN FIXED SO PLEASE DELETE these lines below if you added them on your buildspec file.

---

Before terraform init , add these lines:

  export AWS_ACCESS_KEY_ID=`curl --silent 169.254.170.2:80$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI | jq -r '.AccessKeyId'`
  export AWS_SECRET_ACCESS_KEY=`curl --silent 169.254.170.2:80$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI | jq -r '.SecretAccessKey'`
  export AWS_SESSION_TOKEN=`curl --silent 169.254.170.2:80$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI | jq -r '.Token'`

It is more readable.

Answer 2

In you buildspec.yml try:

env:
  variables:
    AWS_METADATA_ENDPOINT: "http://169.254.169.254:80$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI"

You need this is because TF will look for the meta data in the env var that is not set in the container.

Answer 3

I hate to post this but it will allow terraform to access the codebuild IAM STS access keys and execute terraform commands from within codebuild as a buildspec.yml

It's pretty handy for automated deploys of AWS infrastructure as you can drop a CodeBuild into all your AWS accounts and fire them with a CodePipeline.

Please note the version: 0.2 This passes envars between commands where as version 0.1 had a clean shell for each command

Please update if you find something better:

version: 0.2
env:
  variables:
    AWS_DEFAULT_REGION: "us-west-2"
phases:
  install:
    commands:
      - apt-get -y update
      - apt-get -y install jq
  pre_build:
      commands:

      # load acs submodule (since codebuild doesn't pull the .git folder from the repo
      - cd common 
      - git clone https://gituser@gitlab.com/aws-account-tools/acs.git
      - cd ../

      #install terraform
      - other/install-tf-linux64.sh
      - terraform --version

      #set env variables for terraform provider
      - curl 169.254.170.2$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI | jq 'to_entries | [ .[] | select(.key | (contains("Expiration") or contains("RoleArn"))  | not) ] |  map(if .key == "AccessKeyId" then . + {"key":"AWS_ACCESS_KEY_ID"} else . end) | map(if .key == "SecretAccessKey" then . + {"key":"AWS_SECRET_ACCESS_KEY"} else . end) | map(if .key == "Token" then . + {"key":"AWS_SESSION_TOKEN"} else . end) | map("export \(.key)=\(.value)") | .[]' -r > /tmp/cred.txt # work around https://github.com/hashicorp/terraform/issues/8746
      - chmod +x /tmp/cred.txt
      - . /tmp/cred.txt
  build:
    commands:
      - ls
      - cd your/repo's/folder/with/main.tf 
      - terraform init 
      - terraform plan 
      - terraform apply

Answer 4

Terraform AWS provider offers the following method of authentication:

Static credentials

In this case you can add the access and secrete keys directly into the tf config file as follow:

provider "aws" {
  region     = "us-west-2"
  access_key = "anaccesskey"
  secret_key = "asecretkey"
}

Environment variables

You import the access and secrete key into the the environment variable. Do this using the export command

$ export AWS_ACCESS_KEY_ID="anaccesskey"
$ export AWS_SECRET_ACCESS_KEY="asecretkey"

Shared Credentials file

If Terraform fail to detect credentials inline, or in the environment, Terraform will check this location, $HOME/.aws/credentials in which case you don't need to mention or put the credential in your Terraform config

EC2 Role

If you're running Terraform from an EC2 instance with IAM Instance Profile using IAM Role, Terraform will just ask the metadata API endpoint for credentials. In which case, you don't have to mention the access and secrete keys in any config. This is the preferred way

https://www.terraform.io/docs/providers/aws/ http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html#instance-metadata-security-credentials