How to convert/migrate existing google cloud platform infrastructure to terraform or other IaC

Question

Currently we have our kubernetes cluster master set to zonal, and require it to be regional. My idea is to convert the existing cluster and all workloads/nodes/resources to some infrastructure-as-code - preferably terraform (but could be as simple as a set of gcloud commands).

I know with GCP I can generate raw command lines for commands I'm about to run, but I don't know how (or if I even can) to convert existing infrastructure to the same.

Based on my research, it looks like it isn't exactly possible to do what I'm trying to do [in a straight-forward fashion]. So I'm looking for any advice, even if it's just to read some other documentation (for a tool I'm not familiar with maybe).

TL;DR: I'm looking to take my existing Google Cloud Platform Kubernetes cluster and rebuild it in order to change the location type from zonal to master - I don't actually care how this is done. What is a currently accepted best-practice way of doing this? If there isn't one, what is a quick and dirty way of doing this?

If you require me to specify further, I will - I have intentionally left out linking to specific research I've done.

Answer 1

Creating a Kubernetes cluster with terraform is very straightforward because ultimately making a Kubernetes cluster in GKE is straightforward, you'd just use the google_container_cluster and google_container_node_pool resources, like so:

resource "google_container_cluster" "primary" {
  name               = "${var.name}"
  region             = "${var.region}"
  project            = "${var.project_id}"
  min_master_version = "${var.version}"

  addons_config {
    kubernetes_dashboard {
      disabled = true
    }
  }

  maintenance_policy {
    daily_maintenance_window {
      start_time = "03:00"
    }
  }

  lifecycle {
    ignore_changes = ["node_pool"]
  }

  node_pool {
    name = "default-pool"
  }
}

resource "google_container_node_pool" "default" {
  name    = "default"
  project = "${var.project_id}"
  region  = "${var.region}"
  cluster = "${google_container_cluster.primary.name}"

  autoscaling {
    min_node_count = "${var.node_pool_min_size}"
    max_node_count = "${var.node_pool_max_size}"
  }

  management {
    auto_repair  = "${var.node_auto_repair}"
    auto_upgrade = "${var.node_auto_upgrade}"
  }

  lifecycle {
    ignore_changes = ["initial_node_count"]
  }

  node_config {
    machine_type = "${var.node_machine_type}"

    oauth_scopes = [
      "https://www.googleapis.com/auth/cloud-platform",
    ]
  }

  depends_on = ["google_container_cluster.primary"]
}

For a more fully featured experience, there are terraform modules available like this one

Converting an existing cluster is considerably more fraught. If you want to use terraform import

terraform import google_container_cluster.mycluster us-east1-a/my-cluster

However, in your comment , you mentioned wanting to convert a zonal cluster to a regional cluster. Unfortunately, that's not possible at this time

You decide whether your cluster is zonal or regional when you create it. You cannot convert an existing zonal cluster to regional, or vice versa.

Your best bet, in my opinion, is to:

• Create a regional cluster with terraform, giving the cluster a new name
• Backup your existing zonal cluster, either using an etcd backup, or a more sophisticated backup using heptio-ark
• Restore that backup to your regional cluster

Answer 2

I wanted to achieve exactly that: Take existing cloud infrastructure and bring it to infrastructure as code (IaC), ie put it in *.tf files

There were basically 2 options that I found and took into consideration:

1. terraform import ( Documentation )

   Because of the following limitation terraform import did not achieve exactly what I was looking for, because it requires to manually create the resources.

   The current implementation of Terraform import can only import resources into the state. It does not generate configuration. A future version of Terraform will also generate configuration.

   Because of this, prior to running terraform import it is necessary to write manually a resource configuration block for the resource, to which the imported object will be mapped.

2. Terraformer ( GitHub Repo )

   A CLI tool that generates tf/json and tfstate files based on existing infrastructure (reverse Terraform).

   This tools is provider-agnostic and follows the flow as terraform, ie plan and import . It was able to import specific resources entire workspaces and convet it into *.tf files.