AKS 中的 Cert-manager ClusterIssuer 问题

Question

I am getting this error in clusterissuer (cert-manager version 1.7.1):

"Error getting keypair for CA issuer: error decoding certificate PEM block"

I have the ca.crt, tls.crt and tls.key stored in a Key Vault in Azure.

kubectl describe clusterissuer ca-issuer

  Ca:
    Secret Name:  cert-manager-secret
Status:
  Conditions:
    Last Transition Time:  2022-02-25T11:40:49Z
    Message:               Error getting keypair for CA issuer: error decoding certificate PEM block
    Observed Generation:   1
    Reason:                ErrGetKeyPair
    Status:                False
    Type:                  Ready
Events:
  Type     Reason         Age                  From          Message
  ----     ------         ----                 ----          -------
  Warning  ErrGetKeyPair  3m1s (x17 over 58m)  cert-manager  Error getting keypair for CA issuer: error decoding certificate PEM block
  Warning  ErrInitIssuer  3m1s (x17 over 58m)  cert-manager  Error initializing issuer: error decoding certificate PEM block

kubectl get clusterissuer

NAME        READY   AGE
ca-issuer   False   69m 

• This is the clusterissuer yaml file:

ca-issuer.yaml

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: ca-issuer
  namespace: cert-manager
spec:
  ca:
    secretName: cert-manager-secret

This is the KeyVault yaml file to retrieve the ca.crt, tls.crt and tls.key

keyvauls.yaml

apiVersion: spv.no/v2beta1
kind: AzureKeyVaultSecret
metadata:
  name: secret-akscacrt
  namespace: cert-manager
spec:
  vault:
    name: kv-xx # name of key vault
    object:
      name: akscacrt # name of the akv object
      type: secret # akv object type
  output: 
    secret: 
      name: cert-manager-secret # kubernetes secret name
      dataKey: ca.crt # key to store object value in kubernetes secret
---
apiVersion: spv.no/v2beta1
kind: AzureKeyVaultSecret
metadata:
  name: secret-akstlscrt
  namespace: cert-manager
spec:
  vault:
    name: kv-xx # name of key vault
    object:
      name: akstlscrt # name of the akv object
      type: secret # akv object type
  output: 
    secret: 
      name: cert-manager-secret # kubernetes secret name
      dataKey: tls.crt # key to store object value in kubernetes secret
---
apiVersion: spv.no/v2beta1
kind: AzureKeyVaultSecret
metadata:
  name: secret-akstlskey
  namespace: cert-manager
spec:
  vault:
    name: kv-xx # name of key vault
    object:
      name: akstlskey # name of the akv object
      type: secret # akv object type
  output: 
    secret: 
      name: cert-manager-secret # kubernetes secret name
      dataKey: tls.key # key to store object value in kubernetes secret
---

and these are the certificates used:

---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: argocd-xx
  namespace: argocd
spec:
  secretName: argocd-xx
  issuerRef:
    name: ca-issuer
    kind: ClusterIssuer
  commonName: "argocd.xx"
  dnsNames:
    - "argocd.xx"
  privateKey:
    size: 4096
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: sonarqube-xx
  namespace: sonarqube
spec:
  secretName: "sonarqube-xx"
  issuerRef:
    name: ca-issuer
    kind: ClusterIssuer
  commonName: "sonarqube.xx"
  dnsNames:
    - "sonarqube.xx"
  privateKey:
    size: 4096

I can see that I can retrive the secrets for the certificate from key vault:

kubectl get secret -n cert-manager cert-manager-secret -o yaml

apiVersion: v1
data:
  ca.crt: XXX
  tls.crt: XXX
  tls.key: XXX

Also, another strange thing is that I am getting other secrets in sonarqube/argocd namespace which I deployed previously but are not any more in my deployment file. I cannot delete them, when I try to delete them, they are re-created automatically. Looks like they are stored in some kind of cache. Also I tried to delete the namespace akv2k8s/cert-manager and delete the cert-manager/akv2k8s controllers and re-install them again but same issue after re-installing and applying the deployment...

kubectl get secret -n sonarqube

NAME                                      TYPE                                  DATA   AGE
cert-manager-secret                       Opaque                                3      155m
default-token-c8b86                       kubernetes.io/service-account-token   3      2d1h
sonarqube-xx-7v7dh   Opaque                                1      107m
sql-db-secret                             Opaque                                2      170m

kubectl get secret -n argocd   
NAME                                   TYPE                                  DATA   AGE
argocd-xx-7b5kb   Opaque                                1      107m
cert-manager-secret-argo               Opaque                                3      157m
default-token-pjb4z                    kubernetes.io/service-account-token   3      3d15h

kubectl describe certificate sonarqube-xxx -n sonarqube

Status:
  Conditions:
    Last Transition Time:        2022-02-25T11:04:08Z
    Message:                     Issuing certificate as Secret does not exist
    Observed Generation:         1
    Reason:                      DoesNotExist
    Status:                      False
    Type:                        Ready
    Last Transition Time:        2022-02-25T11:04:08Z
    Message:                     Issuing certificate as Secret does not exist
    Observed Generation:         1
    Reason:                      DoesNotExist
    Status:                      True
    Type:                        Issuing
  Next Private Key Secret Name:  sonarqube-xxx-7v7dh
Events:                          <none>

Any idea?

Thanks.

Answer 1

I figured it out just uploading the certificate info ca.crt . tls.crt and tls.key in plain text, without BASE64 encoding in the Key Vault secrets in Azure.

When AKV2K8S retrives the secrets from the Key Vault and stored in Kube.netes, automatically it is encoded in BASE64.

Regards,