Azure 管道 - terratest - 錯誤：請運行“az login”來設置帳戶

Question

我在 Azure 管道中面臨一個（接縫）經常性 pbm 來運行 terratest。

雖然資源被很好地創建被破壞，但當我調用 azure.ResourceGroupExists function （或其他任何 azure.xxx 函數）時，我有以下錯誤：

--- FAIL: TestTerraform_RM_resource_group (102.30s)
    resourcegroup.go:15: 
            Error Trace:    resourcegroup.go:15
                                        RM_resource_group_test.go:108
            Error:          Received unexpected error:
                            Invoking Azure CLI failed with the following error: ERROR: Please run 'az login' to setup account.
            Test:           TestTerraform_RM_resource_group
FAIL

關於一些論壇，這似乎是一些配置問題，我遵循所有這些推薦的配置：

• 為 terraform 設置環境變量： -- ARM_CLIENT_ID -- ARM_CLIENT_SECRET -- ARM_SUBSCRIPTION_ID -- ARM_TENANT_ID
• 在 terratest 的 go 任務之外的 AzureCli 任務中設置 az login，因為 terratest 似乎需要 2 個不同的身份驗證：（為此 az 登錄使用服務主體客戶端 ID）
• 對於 Assert 測試，需要 ARM_CLIENT 身份驗證
• 對於 Exists 測試，需要服務連接認證

這里是我關注的鏈接：

• https://github.com/gruntwork-io/terratest/issues/454
• https://github.com/gruntwork-io/terratest/tree/master/examples/azure#review-environment-variables
• https://github.com/gruntwork-io/terratest/blob/master/modules/environment/envvar.go
• https://blog.jcorioland.io/archives/2019/09/25/terraform-microsoft-azure-ci-docker-azure-pipeline.html

波紋管我的管道代碼，其中 TF_VAR_ARM_CLIENT_SECRET 是管道的秘密變量

runOnce:
  deploy:
    steps:
    - checkout: self

    - task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
      displayName: 'Install Terraform $(TERRAFORM_VERSION)'
      inputs:
        terraformVersion: $(TERRAFORM_VERSION)

    - task: GoTool@0
      displayName: 'Use Go $(GOVERSION)'
      inputs:
        version: $(GOVERSION)
        goPath: $(GOPATH)
        goBin: $(GOBIN)

    - task: Go@0
      displayName: 'Install Go Terratest module'
      inputs:
        command: get
        arguments: '$(TF_LOG) github.com/gruntwork-io/terratest/modules/terraform'

    - task: Go@0
      displayName: 'Install Go Assert module'
      inputs:
        command: get
        arguments: '$(TF_LOG) github.com/stretchr/testify/assert'

    - task: Go@0
      displayName: 'Install Go Terratest Azure module'
      inputs:
        command: get
        arguments: '$(TF_LOG) github.com/gruntwork-io/terratest/modules/azure'

    - task: Go@0
      displayName: 'Install Go hashicorp/terraform-json module'
      inputs:
        command: get
        arguments: '$(TF_LOG) github.com/hashicorp/terraform-json'

    - task: Go@0
      displayName: 'Install Go azure-sdk-for-go module'
      inputs:
        command: get
        arguments: '$(TF_LOG) github.com/Azure/azure-sdk-for-go'

    - task: AzureCLI@2
      displayName: Azure CLI
      inputs:
        azureSubscription: $(serviceConnection)
        scriptType: ps
        scriptLocation: inlineScript
        inlineScript: |
          az login --service-principal --username $(TF_VAR_ARM_CLIENT_ID) --password $(TF_VAR_ARM_CLIENT_SECRET) --tenant 'f5ff14e7-93c8-49f7-9706-7beea059bd32'

    # Go test command
    - task: Go@0
      displayName: 'Run Go terratest for resource_Modules'
      inputs:
        command: test
        arguments: '$(TF_LOG) $(pathToTerraformRootModule)\resource_group\'
      env:
        ARM_CLIENT_SECRET: $(TF_VAR_ARM_CLIENT_SECRET) #pipeline secret variable
        ARM_CLIENT_ID: $(TF_VAR_ARM_CLIENT_ID)
        ARM_SUBSCRIPTION_ID: $(TF_VAR_ARM_SUBSCRIPTION_ID)
        ARM_TENANT_ID: $(TF_VAR_ARM_TENANT_ID)
        TF_VAR_SERVICE_PRINCIPAL_ID: $(TF_VAR_ARM_CLIENT_ID)
        TF_VAR_SERVICE_PRINCIPAL_SECRET: $(TF_VAR_ARM_CLIENT_ID)
        resource_group_name: $(storageAccountResourceGroup)
        storage_account_name: $(storageAccount)
        container_name: $(stateBlobContainer)
        key: '$(MODULE)-$(TF_VAR_APPLICATION)-${{ parameters.Environment }}.tfstate'

波紋管我的 go terratest 代碼：

package RM_resource_group_Test

import (
    "testing"
    "os"

    "github.com/gruntwork-io/terratest/modules/azure"
    "github.com/gruntwork-io/terratest/modules/terraform"
    "github.com/stretchr/testify/assert"
)

var (
    globalBackendConf = make(map[string]interface{})
    globalEnvVars = make(map[string]string)
)

func TestTerraform_RM_resource_group(t *testing.T) {
    t.Parallel()

    // terraform Directory
    fixtureFolder := "./"
    
    // input value
    inputStage       := "demo_we"
    inputEnvironment := "DEMO"
    inputApplication := "DEMO"

    // expected value
    expectedName := "z-adf-ftnd-shrd-dm-ew1-rgp42"


    // getting enVars from environment variables
    ARM_CLIENT_ID := os.Getenv("ARM_CLIENT_ID")
    ARM_CLIENT_SECRET := os.Getenv("ARM_CLIENT_SECRET")
    ARM_SUBSCRIPTION_ID := os.Getenv("ARM_SUBSCRIPTION_ID")
    ARM_TENANT_ID := os.Getenv("ARM_TENANT_ID")


    if ARM_CLIENT_ID != "" {
        globalEnvVars["ARM_USE_MSI"] = "false"
        globalEnvVars["ARM_CLIENT_ID"] = ARM_CLIENT_ID
        globalEnvVars["ARM_CLIENT_SECRET"] = ARM_CLIENT_SECRET
        globalEnvVars["ARM_SUBSCRIPTION_ID"] = ARM_SUBSCRIPTION_ID
        globalEnvVars["ARM_TENANT_ID"] = ARM_TENANT_ID
    }


    // getting backend vars from environment variables
    resource_group_name := os.Getenv("resource_group_name")
    storage_account_name := os.Getenv("storage_account_name")
    container_name := os.Getenv("container_name")
    key := os.Getenv("key")


    if resource_group_name != "" {
        globalBackendConf["use_msi"] = false
        globalBackendConf["resource_group_name"] = resource_group_name
        globalBackendConf["storage_account_name"] = storage_account_name
        globalBackendConf["container_name"] = container_name
        globalBackendConf["key"] = key
    }
    
    // User Terratest to deploy the infrastructure
    terraformOptions := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
        // The path to where our Terraform code is located
        TerraformDir: fixtureFolder,
        // Variables to pass to our Terraform code using -var options
        Vars: map[string]interface{}{
            "STAGE": inputStage,
            "ENVIRONMENT": inputEnvironment,
            "APPLICATION" : inputApplication,
        },


        EnvVars: globalEnvVars,

        // backend values to set when initialziing Terraform
        BackendConfig: globalBackendConf,
        
        // Disable colors in Terraform commands so its easier to parse stdout/stderr
        NoColor: true,

    })



    // website::tag::4::Clean up resources with "terraform destroy". Using "defer" runs the command at the end of the test, whether the test succeeds or fails.
    // At the end of the test, run `terraform destroy` to clean up any resources that were created
    defer terraform.Destroy(t, terraformOptions)

    // website::tag::2::Run "terraform init" and "terraform apply".
    // This will run `terraform init` and `terraform apply` and fail the test if there are any errors
    terraform.InitAndApply(t, terraformOptions)
    actualName := terraform.Output(t, terraformOptions, "tested_name")
    actualReaderName := terraform.Output(t, terraformOptions, "tested_readerName")
    assert.Equal(t, expectedName, actualName)
    assert.Equal(t, expectedName, actualReaderName)
    
    subscriptionID :=  terraform.Output(t, terraformOptions, "current_subscription_id")
    exists := azure.ResourceGroupExists(t, expectedName, subscriptionID)
    assert.True(t, exists, "Resource group does not exist")
}

在 Azure 中創建和銷毀資源后，我確定我在傳遞參數時遺漏了一些東西，一如既往地出現以下錯誤：

--- FAIL: TestTerraform_RM_resource_group (90.75s)
resourcegroup.go:15: 
        Error Trace:    resourcegroup.go:15
                                    RM_resource_group_test.go:108
        Error:          Received unexpected error:
                        Invoking Azure CLI failed with the following error: ERROR: Please run 'az login' to setup account.
        Test:           TestTerraform_RM_resource_group

請幫忙。

Answer 1

並感謝您的回答..

正如我之前發現的，這是一個配置錯誤，在對 Go Terratest Azure 模塊進行了一些深入挖掘之后，我發現這些行給出了所有解釋：

• https://github.com/gruntwork-io/terratest/blob/master/modules/azure/authorizer.go#L11導致
• https://docs.microsoft.com/en-us/azure/developer/go/azure-sdk-authorization#use-environment-based-authentication

所以我把我的管道改成這樣：

# Go test command
- task: Go@0
  displayName: 'Run Go terratest for resource_Modules'
  inputs:
    command: test
    arguments: '$(TF_LOG) $(pathToTerraformRootModule)\...'
  env:
    ARM_SUBSCRIPTION_ID: $(TF_VAR_ARM_SUBSCRIPTION_ID)
    AZURE_CLIENT_ID: $(TF_VAR_ARM_CLIENT_ID)
    AZURE_TENANT_ID: $(TF_VAR_ARM_TENANT_ID)
    AZURE_CLIENT_SECRET: $(TF_VAR_ARM_CLIENT_SECRET)
    resource_group_name: $(storageAccountResourceGroup)
    storage_account_name: $(storageAccount)
    container_name: $(stateBlobContainer)
    key: '$(MODULE)-$(TF_VAR_APPLICATION)-${{ parameters.Environment }}.tfstate'

還有我的 Go 代碼（關於 envVariables 的使用）：

// getting enVars from environment variables
ARM_CLIENT_ID := os.Getenv("AZURE_CLIENT_ID")
ARM_CLIENT_SECRET := os.Getenv("AZURE_CLIENT_SECRET")
ARM_TENANT_ID := os.Getenv("AZURE_TENANT_ID")
ARM_SUBSCRIPTION_ID := os.Getenv("ARM_SUBSCRIPTION_ID")

// creating globalEnVars for terraform call through Terratest
if ARM_CLIENT_ID != "" {
    //globalEnvVars["ARM_USE_MSI"] = "true"
    globalEnvVars["ARM_CLIENT_ID"] = ARM_CLIENT_ID
    globalEnvVars["ARM_CLIENT_SECRET"] = ARM_CLIENT_SECRET
    globalEnvVars["ARM_SUBSCRIPTION_ID"] = ARM_SUBSCRIPTION_ID
    globalEnvVars["ARM_TENANT_ID"] = ARM_TENANT_ID
}


// getting backend vars from environment variables
resource_group_name := os.Getenv("resource_group_name")
storage_account_name := os.Getenv("storage_account_name")
container_name := os.Getenv("container_name")
key := os.Getenv("key")

// creating globalBackendConf for terraform call through Terratest
if resource_group_name != "" {
    //globalBackendConf["use_msi"] = true
    globalBackendConf["resource_group_name"] = resource_group_name
    globalBackendConf["storage_account_name"] = storage_account_name
    globalBackendConf["container_name"] = container_name
    globalBackendConf["key"] = key
}

// User Terratest to deploy the infrastructure
terraformOptions := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
    // website::tag::1::Set the path to the Terraform code that will be tested.
    // The path to where our Terraform code is located
    TerraformDir: fixtureFolder,
    // Variables to pass to our Terraform code using -var options
    Vars: map[string]interface{}{
        "STAGE": inputStage,
        "ENVIRONMENT": inputEnvironment,
        "APPLICATION" : inputApplication,
        //"configuration" : inputConfiguration,
    },


    // globalvariables for user account 
    EnvVars: globalEnvVars,

    // backend values to set when initialziing Terraform
    BackendConfig: globalBackendConf,
    
    // Disable colors in Terraform commands so its easier to parse stdout/stderr
    NoColor: true,

})

一切順利。 希望這可以幫助其他人。

再次感謝。

[編輯] 更明確地說：

Go 和 Terraform 使用兩種不同的方法進行 Azure 身份驗證。

** Terraform 認證解釋如下：

• https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/service_principal_client_secret#configuring-the-service-principal-in-terraform

** Go 認證解釋如下：

• https://docs.microsoft.com/en-us/azure/developer/go/azure-sdk-authorization#use-environment-based-authentication

** Terratest 使用兩種身份驗證方法來處理它必須完成的工作：

• azure 存在測試使用 Go azure 認證：
  • https://github.com/gruntwork-io/terratest/blob/master/modules/azure/authorizer.go#L11
• terraform 命令使用 terraform 認證：
  • https://github.com/gruntwork-io/terratest/blob/0d654bd2ab781a52e495f61230cf892dfba9731b/modules/terraform/cmd.go#L12
  • https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/service_principal_client_secret#configuring-the-service-principal-in-terraform

所以兩種認證方法都必須實現