terraform how to properly implement delay with time_sleep resource

Question

A resource created by terraform after its creation comsumes CPU/RAM on cluster where it is created, so some kind of delay is needed before the next resource on the same cluster is created.

As an option to achieve this it was decided to use time_sleep terraform resource to implement some delay before resources creation.

It was also decided to use -parallelism=1 so that resources were created one by one.

Let's say we have a module (as simple as it could be):

module test, main.tf

resource "time_sleep" "wait_3_seconds" {
  create_duration = "3s"
}

resource "null_resource" "topic_events" {
  triggers = {
    always_run = timestamp()
    topic = var.topic_name
  }
  depends_on = [time_sleep.wait_3_seconds]
}

module test, variables.tf

variable topic_name {}

Main module invokes module test (see above):

module "test" {
  for_each= tomap(var.environments[var.dim_arr].clusters.events.topics)
  source = "./test"
  topic_name = "${var.dim_arr}.${each.value.topic}"
}

The logic is that input values are handled in the loop but because of the time_sleep resource in the test module some delay is introduced to this loop, that in turn should decrease a load to the server.

However terraform tries to create all time_sleep resources in the nested module and then inerates through the objects in the main module and creates them this way:

a) all time_sleep resources are created

b) all resources that depend on them (see a)) are created

Is there any way to change this logic, so that

-resouce that uses time sleep is created

-time_sleep resource causes delay to decrease a load on the cluster

-next object in the loop is handled

Any suggestions or ideas are appreciated.

Thank you.

Answer 1

The only way I found is to use an external source to determine if it is time to run, if not we wait and all other resources in the module will have to depend on this wait or on other resources that depend on the wait...

I have the code here:
https://github.com/heldersepu/hs-scripts/tree/master/TerraForm/sequential_wait

The idea is to create a file that will keep count for us of who is running now and everyone else waits, I'm creating the file using a null resource:

resource "null_resource" "set_initial_state" {
  provisioner "local-exec" {
    interpreter = ["bash", "-c"]
    command = "echo \"0\" > counter"
  }
}

Then inside your module we check that file and wait:

resource "null_resource" "wait" {
  provisioner "local-exec" {
    interpreter = ["bash", "-c"]
    command = "while [[ $(cat counter) != \"${var.index}\" ]]; do sleep 5; done; sleep 3;"
  }
}

Here is the output of a terraform apply

module.test["3"].null_resource.wait: Creating...
module.test["0"].null_resource.wait: Creating...
module.test["2"].null_resource.wait: Creating...
null_resource.set_initial_state: Creating...
module.test["1"].null_resource.wait: Creating...
module.test["2"].null_resource.wait: Provisioning with 'local-exec'...
module.test["0"].null_resource.wait: Provisioning with 'local-exec'...
module.test["2"].null_resource.wait (local-exec): Executing: ["bash" "-c" "while [[ $(cat counter) != \"2\" ]]; do sleep 5; done; sleep 3;"]
module.test["3"].null_resource.wait: Provisioning with 'local-exec'...
module.test["3"].null_resource.wait (local-exec): Executing: ["bash" "-c" "while [[ $(cat counter) != \"3\" ]]; do sleep 5; done; sleep 3;"]
null_resource.set_initial_state: Provisioning with 'local-exec'...
null_resource.set_initial_state (local-exec): Executing: ["bash" "-c" "echo \"0\" > counter"]
module.test["0"].null_resource.wait (local-exec): Executing: ["bash" "-c" "while [[ $(cat counter) != \"0\" ]]; do sleep 5; done; sleep 3;"]
null_resource.set_initial_state: Creation complete after 0s [id=8614789881521489088]
module.test["1"].null_resource.wait: Provisioning with 'local-exec'...
module.test["1"].null_resource.wait (local-exec): Executing: ["bash" "-c" "while [[ $(cat counter) != \"1\" ]]; do sleep 5; done; sleep 3;"]
module.test["0"].null_resource.wait: Creation complete after 3s [id=2055094284904046261]
module.test["0"].null_resource.test: Creating...
module.test["0"].null_resource.test: Provisioning with 'local-exec'...
module.test["0"].null_resource.test (local-exec): Executing: ["bash" "-c" "date"]
module.test["0"].null_resource.test (local-exec): Tue Nov 30 14:02:16 EST 2021
module.test["0"].null_resource.test: Creation complete after 0s [id=4696397823999126287]
module.test["0"].null_resource.inc: Creating...
module.test["0"].null_resource.inc: Provisioning with 'local-exec'...
module.test["0"].null_resource.inc (local-exec): Executing: ["bash" "-c" "echo \"1\" > counter"]
module.test["0"].null_resource.inc: Creation complete after 0s [id=6621987075727255534]
module.test["1"].null_resource.wait: Creation complete after 8s [id=8925579995767074961]
module.test["1"].null_resource.test: Creating...
module.test["1"].null_resource.test: Provisioning with 'local-exec'...
module.test["1"].null_resource.test (local-exec): Executing: ["bash" "-c" "date"]
module.test["1"].null_resource.test (local-exec): Tue Nov 30 14:02:21 EST 2021
module.test["1"].null_resource.test: Creation complete after 0s [id=5320532609227697061]
module.test["1"].null_resource.inc: Creating...
module.test["1"].null_resource.inc: Provisioning with 'local-exec'...
module.test["1"].null_resource.inc (local-exec): Executing: ["bash" "-c" "echo \"2\" > counter"]
module.test["1"].null_resource.inc: Creation complete after 0s [id=5315818672844469230]
module.test["2"].null_resource.wait: Still creating... [10s elapsed]
module.test["3"].null_resource.wait: Still creating... [10s elapsed]
module.test["2"].null_resource.wait: Creation complete after 13s [id=3209055618166905619]
module.test["2"].null_resource.test: Creating...
module.test["2"].null_resource.test: Provisioning with 'local-exec'...
module.test["2"].null_resource.test (local-exec): Executing: ["bash" "-c" "date"]
module.test["2"].null_resource.test (local-exec): Tue Nov 30 14:02:26 EST 2021
module.test["2"].null_resource.test: Creation complete after 0s [id=4919718470589558801]
module.test["2"].null_resource.inc: Creating...
module.test["2"].null_resource.inc: Provisioning with 'local-exec'...
module.test["2"].null_resource.inc (local-exec): Executing: ["bash" "-c" "echo \"3\" > counter"]
module.test["2"].null_resource.inc: Creation complete after 0s [id=1455157426309278256]
module.test["3"].null_resource.wait: Creation complete after 18s [id=8455991137394172301]
module.test["3"].null_resource.test: Creating...
module.test["3"].null_resource.test: Provisioning with 'local-exec'...
module.test["3"].null_resource.test (local-exec): Executing: ["bash" "-c" "date"]
module.test["3"].null_resource.test (local-exec): Tue Nov 30 14:02:31 EST 2021
module.test["3"].null_resource.test: Creation complete after 0s [id=3412367143530453609]
module.test["3"].null_resource.inc: Creating...
module.test["3"].null_resource.inc: Provisioning with 'local-exec'...
module.test["3"].null_resource.inc (local-exec): Executing: ["bash" "-c" "echo \"4\" > counter"]
module.test["3"].null_resource.inc: Creation complete after 0s [id=4184091859253605007]

Apply complete! Resources: 13 added, 0 changed, 0 destroyed.