google cloud platform instance in MIG cannot access artifact registry

Question

I'm trying to deploy a managed instance group with a load balancer which will be running a web server container. The container is stored in the google artifcat registry.

If I manually create a VM and define the container usage, it is successfully able to pull and activate the container.

When I try to create the managed instance group via terraform, the VM does not pull nor activate the container. When I ssh to the VM and try to manually pull the container, I get the following error:

Error response from daemon: Get https://us-docker.pkg.dev/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

The only notable difference between the VM I created manually to the VM created by terraform is that the manual VM has an external IP address. Not sure if this matters and not sure how to add one to the terraform file.

Below is my main.tf file. Can anyone tell me what I'm doing wrong?

terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
       version = "3.53.0"
    }
     google-beta = {
        source  = "hashicorp/google-beta"
        version = "~> 4.0"
    }
  }
}

provider "google" {
  credentials = file("compute_lab2-347808-dab33a244827.json")

  project = "lab2-347808"
  region  = "us-central1"
  zone    = "us-central1-f"
}

locals {
  google_load_balancer_ip_ranges = [
    "130.211.0.0/22",
    "35.191.0.0/16",
  ]
}

module "gce-container" {
  source = "terraform-google-modules/container-vm/google"
  version = "~> 2.0"

  cos_image_name = "cos-stable-77-12371-89-0"

  container = {
    image = "us-docker.pkg.dev/lab2-347808/web-server-repo/web-server-image"

    volumeMounts = [
      {
        mountPath = "/cache"
        name      = "tempfs-0"
        readOnly  = false
      },
    ]
  }
volumes = [
    {
      name = "tempfs-0"

      emptyDir = {
        medium = "Memory"
      }
    },
  ]

  restart_policy = "Always"
}

resource "google_compute_firewall" "rules" {
  project     = "lab2-347808"
  name        = "allow-web-ports"
  network     = "default"
  description = "Opens the relevant ports for the web server"

  allow {
    protocol  = "tcp"
    ports     = ["80", "8080", "5432", "5000", "443"]
  }

  source_ranges = ["0.0.0.0/0"]
  #source_ranges = local.google_load_balancer_ip_ranges
  target_tags = ["web-server-ports"]
}

resource "google_compute_autoscaler" "default" {
  name   = "web-autoscaler"
  zone   = "us-central1-f"
  target = google_compute_instance_group_manager.default.id

  autoscaling_policy {
    max_replicas    = 10
    min_replicas    = 1
    cooldown_period = 60

    cpu_utilization {
      target = 0.5
    }
  }
}
resource "google_compute_instance_template" "default" {
  name           = "my-web-server-template"
  machine_type   = "e2-medium"
  can_ip_forward = false

  tags         = ["ssh", "http-server", "https-server", "web-server-ports"]


  disk {
   #source_image =  "cos-cloud/cos-73-11647-217-0"
    source_image = module.gce-container.source_image
  }

  network_interface {
   network = "default"
  }


  service_account {
    #scopes = ["userinfo-email", "compute-ro", "storage-ro"]
    scopes = ["cloud-platform"]
  }
  metadata = {
    gce-container-declaration = module.gce-container.metadata_value
  }
}

resource "google_compute_target_pool" "default" {
  name = "web-server-target-pool"
}

resource "google_compute_instance_group_manager" "default" {
  name = "web-server-igm"
  zone = "us-central1-f"

  version {
    instance_template  = google_compute_instance_template.default.id
    name               = "primary"
  }

  target_pools       = [google_compute_target_pool.default.id]
  base_instance_name = "web-server-instance"
}

Answer 1

Your VM templates haven't public IPs, therefore, you can't reach public IP.

However, you have 3 ways to solve that issue:

• Add a public IP on the VM template (bad idea)
• Add a Cloud NAT on your VM private IP range to allow outgoing traffic to the internet (good idea)
• Activate the Google private access in the subnet that host the VM private iP range. It create a bridge to access to Google services without having a public IP (my prefered idea) -> https://cloud.google.com/vpc/docs/configure-private-google-access

Answer 2

Apparently I was missing the following acecss_config inside network_interface of the google_compute_instance_template as following:

network_interface {
   network = "default"
   access_config {
     network_tier = "PREMIUM"
   }