AWS Codepipeline using namespaces in Terraform

Question

Has anyone out there worked with namespaces in was Codepipeline? I have the reference link from AWS but as always Terraform does not have actual examples. What I want to do is take the output variable from one buildspec in one phase and access those later in my deploy stage. I feel like I am headed down the right path with the syntax I am using below but I keep getting an error regarding the usage of the namespace.

Stage Build: Snippet from the code

  action {
      run_order         = 1           
      name              = "Terraform-Plan"
      category          = "Build"
      owner             = "AWS"
      provider          = "CodeBuild"
      version           = "1"
      input_artifacts   = ["CodeWorkspace"]
      output_artifacts  = ["CodeSource","TerraformPlanFile"]
      namespace         = "buildvariables" 
    
      configuration = {
        ProjectName          = var.codebuild_project_name
        EnvironmentVariables = jsonencode([
          {
            name  = "CONFIG"
            value = "#{buildvariables.CONFIG}"
            type  = "PLAINTEXT"
          }
        ])
      }

  }

This is the stage where I tried to reference the namespace inside my buildspec_plan file. Not sure i am invoking it correctly:

version: 0.2
env:
  variables:
    ...some variables here

  parameter-store:
    ...some variables here
 
  secrets-manager:
    ...some variables here

  git-credential-helper: yes
    
  exported-variables:
    - CONFIG

phases:
  install:
    commands:
      - ...some commands here
 
  pre_build:
      - ...some commands here

  build:
    on-failure: ABORT
    commands: 
      # Import S3 Folder Location Variable
      - ...some commands here
      - CONFIG=newvariable

I have removed parts of the buildspec which are not relevant to this but my first question is, am I using it correctly because it seems to error out with the following error:

Error creating CodePipeline: InvalidActionDeclarationException: Variables can only reference namespaces produced by an earlier stage or earlier run order in the same stage. The references for the following variables are not valid. StageName=[Plan], ActionName=[Terraform-Plan], ActionConfigurationKey=[EnvironmentVariables], VariableReferenceText=[buildvariables.CONFIG]

I believe i am headed in the right direction but can not find working examples on the terraform usage.

The last part of my question does not relate to my current issue but more with using the variable in another stage once I finally get past the namespace declaration. I plan on invoking this variable in a different build spec known as buildspec_apply by doing the following:

  build:
    commands:
      - cd ${CODEBUILD_SRC_DIR}
      - ls
      - echo ${buildvariables.CONFIG}

I am obviously using an echo as a sample to verify the variable works but i am not sure this is correct.

Links and tips are welcomed thank you

Answer 1

Hello everyone after several days of research I finally got this to work and understand it. For anyone looking for a similar answer here is my best guidance to my own question.

First, remember this part the namespace has to be declared in a previous phase before its use. You can not use the namespace in the same phase you declared it. Equally important is the fact that environment variables are tied directly to the Namespace, and the variables you declare in the namespace phase are tied directly to the namespace. You can not add additional variables in different phases. They must be defined in the phase where the namespace is declared.

Declaring Namespace: Notice the Namespace declaration, it's looking for a string variable. For this example, I used "YOUR_NAME_SPACE_NAME", please replace it with the name you prefer. I would recommend uppercase as much as possible.

stage {
  name = "Plan"
  action {
      run_order         = 1           
      name              = "Terraform-Plan"
      category          = "Build"
      owner             = "AWS"
      provider          = "CodeBuild"
      version           = "1"
      namespace         = "YOUR_NAME_SPACE_NAME" 
      input_artifacts   = ["CodeWorkspace"]
      output_artifacts  = ["CodeSource","TerraformPlanFile"]
    
      configuration = {
        ProjectName          = var.codebuild_project_name_1
        EnvironmentVariables = jsonencode([
          {
            name  = "PL_TESTVARIABLE"
            type  = "PLAINTEXT"  
            value = ""
          },
        ])
        
      }

  }

The key to the Namespace is the environment variable declaration. This happens inside the configuration portion of your phase. Notice that the environment variables have to be set up in JSON. Anyone wondering, I use PL for Pipeline just to avoid a collision with any would-be native variables, feel free to name them any way you want but understand there are native environment variables to Codepipeline. The big takeaway here is that any variable you declare in this phase is now part of the Namespace.

THE BUILDSPEC In the Buildspec of the current phase, you can leverage the Namespace variable to pass values. Please note that in the current stage you can change the value of the Namespace through the Buildspec by declaring a value to the variable.

Note you can declare the variable during the Install, pre-build, or build stage.

    version: 0.2
    env:
      variables:
        ...some variables here
    
      parameter-store:
        ...some variables here
     
      secrets-manager:
        ...some variables here
    
      git-credential-helper: yes
        
      exported-variables:
        - PL_TESTVARIABLE
    
    phases:
      install:
        commands:
          - ...some commands here
     
      pre_build:
          - ...some commands here
    
      build:
        on-failure: ABORT
        commands: 
          # Import S3 Folder Location Variable
          - ...some commands here
          - PL_TESTVARIABLE="MyNewValue"

Notice that in the example we defined the variable in the Export phase and in the Build phase. This is to ensure we output the new value at the end of the build phase.

THE NEXT CODEPIPELINE PHASE: In the next Codepipeline phase, you want to reference the Namespace. To do this you need to declare the variable within the environment section just like in the previous phase and declare the same variable. Keep in mind each phase treats variables as new variables that don't carry over from other phases. To remedy this we will leverage the Namespace.

  stage {
    name = "Test"
      action {
          run_order         = 1           
          name              = "Terraform-Test"
          category          = "Test"
          owner             = "AWS"
          provider          = "CodeBuild"
          version           = "1"
          input_artifacts   = ["CodeSource"]
          output_artifacts  = ["CodeSource"]
        
          configuration = {
            ProjectName          = var.codebuild_project_name_1
            EnvironmentVariables = jsonencode([
              {
                name  = "PL_TESTVARIABLE"
                type  = "PLAINTEXT"  
                value = "#{YOUR_NAME_SPACE_NAME.PL_TESTVARIABLE}"
              },
            ])
            
          }
    
      }

Notice that we declared PL_TESTVARIABLE again but see what we did with the value of the variable. Remember this is json, not actual terraform so variables are treated as json inputs. To reference the namespace value from the previous phase follow this syntax, Namespace.YourVariableName. Notice PL_TESTVARIABLE value follows our syntax to assign it the value from the export from the previous phase.

CURRENT PHASE BUILDSPEC: To call this variable in this stage inside your Buildspec you would just call the environment variable PL_TESTVARIABLE as you would any part of the Buildspec.

version: 0.2
env:
  variables:
    ...some variables here

phases:
  install:
    commands:
      - echo $PL_TESTVARIABLE

  pre_build:
      - echo "text in line variable ${PL_TESTVARIABLE}"
      - PL_TESTVARIABLE="MyNewValue"

  build:
    on-failure: ABORT
    commands: 
      # Import S3 Folder Location Variable
      - PL_TESTVARIABLE="MyNewValue"
      - NewVariable=$PL_TESTVARIABLE

In this Buildspec reference, I demonstrate various ways to use the variable. This explains how to leverage the Namespace. You don't have to reference it in a sequential manner you could leverage the variable at any stage at any point as long as the Namespace is defined.

Hopefully, this helps anyone trying to leverage this in Terraform.

Cheers