Trying to pass secret from Azure KeyVault via Azure pipeline as env var to an app deployed on AKS

Question

I am trying to get a secret named "ApplicationInsightsInstrumentationKey" from an Azure Key Vault and make it available as environment variable to the well-known Microsoft Weather Forecast example ASP.Net Core application , which I deploy to AKS.

So in the Azure pipeline file I have these 2 tasks (the "Namespace" pipeline parameter value is "ccg-afarber-v3" and I use it for the overall secret name too. And the Key Vault is "ccg-afarber-v3-kv"):

- task: AzureKeyVault@2
  displayName: Read secrets from KV
  inputs:
    azureSubscription: $(armConnection)
    KeyVaultName: '${{ parameters.Namespace }}-kv'
    SecretsFilter: '*'
    RunAsPreJob: false

- task: AzureCLI@2
  displayName: Set secrets in AKS
  inputs:
    azureSubscription: $(armConnection)
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: |
      az aks get-credentials --resource-group ccg-config --name ccg-config-cluster
      kubectl delete 'secrets/${{ parameters.Namespace }}'
      kubectl create secret generic ${{ parameters.Namespace }} --from-literal=ApplicationInsightsInstrumentationKey=$(ApplicationInsightsInstrumentationKey)

The tasks succeed and I can see the secret in the AKS:

After that I try to deploy the app via the following pipeline task:

- task: KubernetesManifest@0
  displayName: Deploy to AKS
  inputs:
    action: deploy
    namespace: ${{ parameters.Namespace }}
    kubernetesServiceConnection: $(aksConnection)
    manifests: |
      $(Build.ArtifactStagingDirectory)/manifests/deployment.yml
      $(Build.ArtifactStagingDirectory)/manifests/service.yml
    containers: |
      $(containerRegistry)/$(imageRepository):$(imageTag)

The task is using the deployment.yml file listed below:

metadata:
  name: exampleapplication 
spec:
  replicas: 1
  selector:
    matchLabels:
      app: exampleapplication
  template:
    metadata:
      labels:
        app: exampleapplication 
    spec:
      containers:
        - name: exampleapplication 
          image: ccgconfigregistry.azurecr.io/exampleapplication:3a5e80a684c83e8037c2ccb07a45c8345501d154
          ports:
          - containerPort: 80
          env:
          - name: ApplicationInsightsInstrumentationKey
            valueFrom:
              secretKeyRef:
                name: ccg-afarber-v3
                key: ApplicationInsightsInstrumentationKey
                optional: false

Unfortunately, the deployment is stuck with:

# kubectl get pod --namespace ccg-afarber-v3
NAME                                 READY   STATUS                       RESTARTS   AGE
exampleapplication-c76448f49-hds2s   0/1     CreateContainerConfigError   0          19m

The kubectl describe pod --namespace ccg-afarber-v3 command prints:

Events:
  Type     Reason     Age                 From               Message
  ----     ------     ----                ----               -------
  Normal   Scheduled  20m                 default-scheduler  Successfully assigned ccg-afarber-v3/exampleapplication-c76448f49-hds2s to aks-nodepool1-84765107-vmss000003
  Warning  Failed     18m (x12 over 20m)  kubelet            Error: secret "ccg-afarber-v3" not found
  Normal   Pulled     17s (x97 over 20m)  kubelet            Container image "ccgconfigregistry.azurecr.io/exampleapplication:3a5e80a684c83e8037c2ccb07a45c8345501d154" already present on machine

I keep re-reading the docs:

• https://cloud.google.com/kube.netes-engine/docs/concepts/secret#from_literal_values
• https://kube.netes.io/docs/concepts/configuration/secret/#using-secrets-as-environment-variables

but I don't understand, why is secret "ccg-afarber-v3" not found .

Answer 1

I have found my mistake - I was creating the Kube.netes secret in the "default" namespace, while my app was trying to retrieve it in the "ccg-afarber-v3" namespace.

The following pipeline task with the added --namespace=ccg-afarber-v3 parameters has resolved my problem:

- task: AzureCLI@2
  displayName: Set secrets in AKS
  inputs:
    azureSubscription: $(armConnection)
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: |
      az aks get-credentials --resource-group ccg-config --name ccg-config-cluster
      kubectl delete 'secrets/${{ parameters.Namespace }}' --namespace=${{ parameters.Namespace }}
      kubectl create secret generic ${{ parameters.Namespace }} --from-literal=ApplicationInsightsInstrumentationKey=$(ApplicationInsightsInstrumentationKey) --namespace=${{ parameters.Namespace }}