基于先前配置的 Kube.netes 服务的 AWS ELB 从 CI 动态创建 Route53 托管区域 A 记录

Question

We have a AWS EKS setup ( full repo here ), where we install Traefik using Helm . This creates a Kube.netes Service called traefik which gets provisioned an AWS Elastic Load Balancer. The ELB url can be obtained using kubectl like this:

$ kubectl get service traefik -n default --output=jsonpath='{.status.loadBalancer.ingress[0].hostname}'
ad07f3f3013fc4539811de662a07cf9a-1862511283.eu-central-1.elb.amazonaws.com

We also have a AWS Route53 domain registered, which we want to point to the dynamically provisioned AWS ELB Traefik is beeing configured behind.

Now since this setup is dynamically provisioned using GitHub Actions based on a potentially new EKS cluster (using Pulumi), we cannot hard code the ELB url into our Route53 hosted zone A record. Instead we have to create or update it every time, the Pulumi provisioned EKS setup or the Traefik Service changes the ELB (eg by provisioning a new ELB and pruning the old).

So how can we create (and update) the Route53 hosted zone A record dynamically from within GitHub Actions?

Answer 1

We choose to use AWS CLI to do that for us. The docs provide a starting point . But we can't do this using a static file like with the proposed --change-batch file://sample.json - instead we need to have it dynamic so we can use a command inside our GitHub Actions workflow.

The idea is derived from this so answer , where we can simply use the json snippet inline without an extra file. Also need to have an idempotent solution which we can run 1 or many times in GitHub Actions CI. Therefore we used the "Action": "UPSERT" (see https://aws.amazon.com/premiumsupport/knowledge-center/simple-resource-record-route53-cli/ ).

aws route53 change-resource-record-sets \
  --hosted-zone-id $ROUTE53_DOMAIN_HOSTED_ZONE_ID \
  --change-batch '
  {
    "Comment": "Create or update Route53 hosted zone A record to point to ELB Traefik is configured to"
    ,"Changes": [{
      "Action"              : "UPSERT"
      ,"ResourceRecordSet"  : {
        "Name"              : "*.'"$ROUTE53_DOMAIN_NAME"'"
        ,"Type"             : "A"
        ,"AliasTarget": {
            "HostedZoneId": "'"$ELB_HOSTED_ZONE_ID"'",
            "DNSName": "dualstack.'"$ELB_URL"'",
            "EvaluateTargetHealth": true
        }
      }
    }]
  }
  '

Using variables inside the json provided to the --change-batch parameter, we need to use single quotes and open them up immediately after (also see https://stackoverflow.com/a/49228748/4964553 )

As you can see, we need to configure 4 variables to make this command run:

1. $ROUTE53_DOMAIN_HOSTED_ZONE_ID : This is the hosted zone id of your Route53 domain you need to register before (the registration itself is a manual step)
2. $ROUTE53_DOMAIN_NAME : Your Route53 registered domain name. As we want all routing to be done by Traefik, we can configure a wildcard record here using *.$ROUTE53_DOMAIN_NAME
3. $ELB_HOSTED_ZONE_ID : A different hosted zone id than your domain! . This is the hosted zone id of the Elastic Load Balancer, which gets provisioned through the Traefik Service deployment (via Helm).
4. $ELB_URL : The ELB url of the Traefik Service. We need to preface it with dualstack. in order to make it work (see https://docs.aws.amazon.com/Route53/latest/APIReference/API_AliasTarget.html )

Obtaining all those variables isn't trivial. We can start with the Route53 domain name, we need to configure as a static GitHub Actions environment varialbe at the top of our provision.yml:

name: provision

on: [push]

env:
  ...
  ROUTE53_DOMAIN_NAME: tekton-argocd.de
...

      - name: Create or update Route53 hosted zone A record to point to ELB Traefik is configured to
        run: |
          echo "--- Obtaining the Route53 domain's hosted zone id"
          ROUTE53_DOMAIN_HOSTED_ZONE_ID="$(aws route53 list-hosted-zones-by-name | jq --arg name "$ROUTE53_DOMAIN_NAME." -r '.HostedZones | .[] | select(.Name=="\($name)") | .Id')"

          echo "--- Obtaining the ELB hosted zone id"
          echo "Therefore cutting the ELB url from the traefik k8s Service using cut (see https://stackoverflow.com/a/29903172/4964553)"
          ELB_NAME="$(kubectl get service traefik -n default --output=jsonpath='{.status.loadBalancer.ingress[0].hostname}' | cut -d "-" -f 1)"
          echo "Extracting the hosted zone it using aws cli and jq (see https://stackoverflow.com/a/53230627/4964553)"
          ELB_HOSTED_ZONE_ID="$(aws elb describe-load-balancers | jq --arg name "$ELB_NAME" -r '.LoadBalancerDescriptions | .[] | select(.LoadBalancerName=="\($name)") | .CanonicalHostedZoneNameID')"

          echo "--- Obtaining the Elastic Load Balancer url as the A records AliasTarget"
          ELB_URL="$(kubectl get service traefik -n default --output=jsonpath='{.status.loadBalancer.ingress[0].hostname}')"

Having all the variables filled we are able to use AWS CLI to create the Route53 record dynamically:

      echo "--- Creating or updating ('UPSERT') Route53 hosted zone A record to point to ELB Traefik (see https://aws.amazon.com/premiumsupport/knowledge-center/simple-resource-record-route53-cli/)"
      echo "--- Creating Route53 hosted zone record (mind to wrap the variables in double quotes in order to get them evaluated, see https://stackoverflow.com/a/49228748/4964553)"
      aws route53 change-resource-record-sets \
        --hosted-zone-id $ROUTE53_DOMAIN_HOSTED_ZONE_ID \
        --change-batch '
        {
          "Comment": "Create or update Route53 hosted zone A record to point to ELB Traefik is configured to"
          ,"Changes": [{
            "Action"              : "UPSERT"
            ,"ResourceRecordSet"  : {
              "Name"              : "*.'"$ROUTE53_DOMAIN_NAME"'"
              ,"Type"             : "A"
              ,"AliasTarget": {
                  "HostedZoneId": "'"$ELB_HOSTED_ZONE_ID"'",
                  "DNSName": "dualstack.'"$ELB_URL"'",
                  "EvaluateTargetHealth": true
              }
            }
          }]
        }
        '

Running your GitHub Actions workflow should result in the Route53 record beeing created. You can have a look into the AWS console:

Here's a build log and also the full GitHub Actions workflow yaml: https://github.com/jonashackt/tekton-argocd-eks/blob/main/.github/workflows/provision.yml