Dependency Violation when changing/deleting a resource in terraform

Question

I am facing an issue when deploy services to AWS via Terraform . The error I got is:

module.elasticsearch.aws_security_group.es: Still destroying... [id=sg-039f33d9b29a38c46, 9m39s elapsed]
module.elasticsearch.aws_security_group.es: Still destroying... [id=sg-039f33d9b29a38c46, 9m49s elapsed]
module.elasticsearch.aws_security_group.es: Still destroying... [id=sg-039f33d9b29a38c46, 9m59s elapsed]

Error: Error deleting security group: DependencyViolation: resource sg-039f33d9b29a38c46 has a dependent object
    status code: 400, request id: cd20a8e3-231f-444f-8479-b0575a98b390

It tries to delete a security group but found out there is a dependency object which stop it deleting it. I checked on AWS console the security group has an associate network interface which I don't have permission to delete. I found below changes on terraform plan :

~ name                   = "SecuritySearchElasticSearchtest" -> "SecuritySearchElasticSearch" # forces replacement

It tries to rename a security group during deployment.

I wonder why Terraform tries to redeploy it. Why not just use the existing resources? Can I force Terraform to not replace?

Answer 1

You can try to remove that SG from state by making command like

terraform state rm module.elasticsearch.aws_security_group.es

and then you can try to import that resource by making command

terraform import module.elasticsearch.aws_security_group.es <security_group_id>

please also keep in mind that in your terraform code name of security group should be SecuritySearchElasticSearchtest if you want terraform to avoid destroying that SG

Answer 2

First Let's explain the problem you're dealing with

The problem : You're trying to apply some change on a resource in terraform though this resource is associated and connected to another resources. In this case, aws security group id reference is mapped and connected to another security groups and therefore a DependencyViolation error is shown.

For example:

sg_a ( ingress: x, y, z       | outgress:all ) 
sg_b ( ingress: a, b, c, sg_a | outgress:all )

When sg_a is being removed the reference inbound still exists on sg_b and dependent on it. You applying your change and getting timeout message like:

...
module.some_module.aws_security_group: Still destroying... [id=sg-a, 10m elapsed]
module.some_module.aws_security_group: Still destroying... [id=sg-a, 13m elapsed]
module.some_module.aws_security_group: Still destroying... [id=sg-a, 14m elapsed]
module.some_module.aws_security_group: Still destroying... [id=sg-a, 15m elapsed]
 
Error: Error deleting security group: DependencyViolation: resource sg-a has a dependent object
status code: 400, request id: abc-123-zxc-567```

According to the reference the dependency violation timeout comes after 15 minutes.

delete - (Default 15m) How long to retry on DependencyViolation errors during security group deletion from lingering ENIs left by certain AWS services such as Elastic Load Balancing. NOTE: Lambda ENIs can take up to 45 minutes to delete, which is not affected by changing this customizable timeout (in version 2.31.0 and later of the Terraform AWS Provider) unless it is increased above 45 minutes.

Solution : The best way dealing with this violation dependency is refreshing the external resources using and depends on this dependency (in our example sg_b should not use sg_a , this ingress should be removed, afterwards applying your changes on terraform again and sg_a will be deleted.

in our example (remove sg_a ingress reference from sg_b security group):

sg_a ( ingress: x, y, z | outgress:all ) 
sg_b ( ingress: a, b, c | outgress:all )

Now you can run terraform apply :

sg_b ( ingress: a, b, c | outgress:all )