How to create kubernetes cluster in google cloud platform in python using google-cloud-container module

Question

I'm trying to create kubernetes cluster on google cloud platform through python (3.7) using google-cloud-container module.

Created kubernetes cluster through google cloud platform and was able to successfully retrieve details for that cluster using google-cloud container (python module).

I'm trying now to create kubernetes cluster through this module. I created a JSON file with required key values and passed it as parameter, but i'm getting errors. Would appreciate if provided a sample code for creating kubernetes cluster in google cloud platform. Thank you in advance.

from google.oauth2 import service_account
from google.cloud import container_v1



class GoogleCloudKubernetesClient(object):

    def __init__(self, file, project_id, project_name, zone, cluster_id):
        credentials = service_account.Credentials.from_service_account_file(
            filename=file)
        self.client = container_v1.ClusterManagerClient(credentials=credentials)
        self.project_id = project_id
        self.zone = zone

    def create_cluster(self, cluster):
        print(cluster)
        response = self.client.create_cluster(self.project_id, self.zone, cluster=cluster)
        print(f"response for cluster creation: {response}")


def main():
    cluster_data = {
            "name": "test_cluster",
            "masterAuth": {
                "username": "admin",
                "clientCertificateConfig": {
                    "issueClientCertificate": True
                }
            },
            "loggingService": "logging.googleapis.com",
            "monitoringService": "monitoring.googleapis.com",
            "network": "projects/abhinav-215/global/networks/default",
            "addonsConfig": {
                "httpLoadBalancing": {},
                "horizontalPodAutoscaling": {},
                "kubernetesDashboard": {
                    "disabled": True
                },
                "istioConfig": {
                    "disabled": True
                }
            },
            "subnetwork": "projects/abhinav-215/regions/us-west1/subnetworks/default",
            "nodePools": [
                {
                    "name": "test-pool",
                    "config": {
                        "machineType": "n1-standard-1",
                        "diskSizeGb": 100,
                        "oauthScopes": [
                            "https://www.googleapis.com/auth/cloud-platform"
                        ],
                        "imageType": "COS",
                        "labels": {
                            "App": "web"
                        },
                        "serviceAccount": "abhinav@abhinav-215.iam.gserviceaccount.com",
                        "diskType": "pd-standard"
                    },
                    "initialNodeCount": 3,
                    "autoscaling": {},
                    "management": {
                        "autoUpgrade": True,
                        "autoRepair": True
                    },
                    "version": "1.11.8-gke.6"
                }
            ],
            "locations": [
                "us-west1-a",
                "us-west1-b",
                "us-west1-c"
            ],
            "resourceLabels": {
                "stage": "dev"
            },
            "networkPolicy": {},
            "ipAllocationPolicy": {},
            "masterAuthorizedNetworksConfig": {},
            "maintenancePolicy": {
                "window": {
                    "dailyMaintenanceWindow": {
                        "startTime": "02:00"
                    }
                }
            },
            "privateClusterConfig": {},
            "databaseEncryption": {
                "state": "DECRYPTED"
            },
            "initialClusterVersion": "1.11.8-gke.6",
            "location": "us-west1-a"
        }


    kube = GoogleCloudKubernetesClient(file='/opt/key.json', project_id='abhinav-215', zone='us-west1-a')

    kube.create_cluster(cluster_data)


if __name__ == '__main__':
    main()

Actual Output:
Traceback (most recent call last):
  File "/opt/matilda_linux/matilda_linux_logtest/matilda_discovery/matilda_discovery/test/google_auth.py", line 118, in <module>
    main()
  File "/opt/matilda_linux/matilda_linux_logtest/matilda_discovery/matilda_discovery/test/google_auth.py", line 113, in main
    kube.create_cluster(cluster_data)
  File "/opt/matilda_linux/matilda_linux_logtest/matilda_discovery/matilda_discovery/test/google_auth.py", line 31, in create_cluster
    response = self.client.create_cluster(self.project_id, self.zone, cluster=cluster)
  File "/opt/matilda_discovery/venv/lib/python3.6/site-packages/google/cloud/container_v1/gapic/cluster_manager_client.py", line 407, in create_cluster
    project_id=project_id, zone=zone, cluster=cluster, parent=parent
ValueError: Protocol message Cluster has no "masterAuth" field.

Answer 1

Kind of late answer, but I had the same problem and figured it out. Worth writing for future viewers.

You should not write the field names in the cluster_data as they appear in the REST API . Instead you should translate them to how they would look by python convention, with words separated by underline instead of camelcase. Thus, instead of writing masterAuth, you should write master_auth. You should make similar changes to the rest of your fields, and then the script should work.

PS you aren't using the project_name and cluster_id params in GoogleCloudKubernetesClient. init . Not sure what they are, but you should probably remove them.

Answer 2

The module is still using the basic REST API format to create the cluster. You can also use the GUI to choose all the options you want to use for your cluster, then press on the REST hyperlink at the bottom of the page, this will provide you with the REST format required to build the cluster you want.

The error you are getting is because you have a blank (or unspecified) field that must be specified. Some of the fields listed on the API have default values that you don't need, others are required.