如何将Azure AKS Kubernetes Cluster自签名CA添加到GitLab CI / CD Kubernetes集成？

Question

I'm trying to add my Azure AKS Kubernetes cluster to my GitLab CI/CD Kubernetes integration.

I can execute kubectl commands on the cluster from my pc, after I ran this command:

az aks get-credentials --resource-group <resource-group-name> --name <kubernetes-cluster-name>

It created a .kube/config file with a content like this:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <some long base64 string here>
    server: https://<resource-group-name+some-hexadecimal-chars>.hcp.westeurope.azmk8s.io:443
  name: <kubernetes-cluster-name>
contexts:
- context:
    cluster: <kubernetes-cluster-name>
    user: clusterUser_<resource-group-name>_<kubernetes-cluster-name>
  name: <kubernetes-cluster-name>
current-context: <kubernetes-cluster-name>
kind: Config
preferences: {}
users:
- name: clusterUser_<resource-group-name>_<kubernetes-cluster-name>
  user:
    client-certificate-data: <some long base64 string here>
    client-key-data: <some long base64 string here>
    token: <some secret string of hexadecimal chars here>

In GitLab form, I have to input these fields:

1. Kubernetes cluster name
2. API URL
3. CA Certificate - Certificate Authority bundle (PEM format)
4. Token
5. Project namespace (optional, unique)

I tried these values:

1. I put my <kubernetes-cluster-name> to match the name of the cluster on azure and the cluster name on the .kube/config file.
2. I put the url https://<resource-group-name+some-hexadecimal-chars>.hcp.westeurope.azmk8s.io:443 copied from the .kube/config file.
3. I tried first the certificate-authority-data from the .kube/config file, but didn't work and I already tried all three base64 strings from the .kube/config file, none worked.
4. I put the token from the .kube/config file.
5. Leave this empty, as it is optional.

In GitLab, When I try to hit the button Install to install the Helm Tiller, I got this error:

Something went wrong while installing Helm Tiller
Can't start installation process. nested asn1 error

And sometimes I get this error instead:

Kubernetes error: SSL_connect returned=1 errno=0 state=error: certificate verify failed

I'm trying to make this to work since yesterday, had google it a lot and doesn't find anything.

I think the problem is with this 3rd field, the CA Certificate, maybe there are some other way to get this content from the command line az or kubectl .

Are there someone here that already got this Kubernetes integration from GitLab to Azure AKS working?

Answer 1

I found out later that the base64 string in the certificate-authority-data of the .kube/config file that I was coping its content into the CA Certificate field of GitLab "Add Kubernetes cluster" form, it is the PEM format, but base64 encoded.

The PEM format already is a base64 encoded representation of the certificate bits, but it has some line breaks in the middle. This whole content is base64 encoded again before it goes to the .kube/config so it is turned into a big base64 single-line string.

I just had to base64 decode this big single-line string (I used the javascript atob("....") in the Chrome's Console window), what gave me something like this:

-----BEGIN CERTIFICATE-----
MIIEyDCCArCgAwIBAgIRAOL3N8oMIwWIxcFTZhTkfgMwDQYJKoZIhvcNAQELBQAw
...
...
...
5gP7yoL1peZ+AWjCgcUVZYiItqrBLpWYDgY9g8btYDUIiWlqkmC0+kBaPfwCtckx
cUp3vlwRITrv0mzrxiQjTLTUpEy7EcD+U6IecA==
-----END CERTIFICATE-----

Then I just copied this content into the GitLab "CA Certificate" field and it worked.