Kubectl against GKE Cluster through Terraform's local-exec?

Question

I am trying to make an automatic migration of workloads between two node pools in a GKE cluster. I am running Terraform in GitLab pipeline. When new node pool is created the local-exec runs and I want to cordon and drain the old node so that the pods are rescheduled on the new one. I am using this registry.gitlab.com/gitlab-org/terraform-images/releases/1.1:v0.43.0 image for my Gitlab jobs. Also, python3 is installed with apk add as well as gcloud cli - downloading the tar and using the gcloud binary executable from google-cloud-sdk/bin directory. I am able to use commands like ./google-cloud-sdk/bin/gcloud auth activate-service-account --key-file=<key here> .

The problem is that I am not able to use kubectl against my cluster.

Although I have installed the gke-gcloud-auth-plugin with ./google-cloud-sdk/bin/gcloud components install gke-gcloud-auth-plugin --quiet once in the CI job and second time in the local-exec script in HCL code I get the following errors:

module.create_gke_app_cluster.null_resource.node_pool_provisioner (local-exec): E0112 16:52:04.854219 259 memcache.go:238] couldn't get current server API group list: Get "https://<IP>/api?timeout=32s": getting credentials: exec: executable <hidden>/google-cloud-sdk/bin/gke-gcloud-auth-plugin failed with exit code 1

290module.create_gke_app_cluster.null_resource.node_pool_provisioner (local-exec): Unable to connect to the server: getting credentials: exec: executable <hidden>/google-cloud-sdk/bin/gke-gcloud-auth-plugin failed with exit code 1

When I check the version of the gke-gcloud-auth-plugin with gke-gcloud-auth-plugin --version I am getting the following error: 174/bin/sh: eval: line 253: gke-gcloud-auth-plugin: not found Which clearly means that the plugin is not installed.

The image that I am using is based on alpine for which there is no way to install the plugin via package manager, unfortunately.

Edit: gcloud components list shows gke-gcloud-auth-plugin as installed too.

Answer 1

The solution was to use google/cloud-sdk image in which I have installed terraform and used this image for the job in question.