[英]Cannot create many Azure firewall rule sets concurrently in Terraform
My Terraform code is broadly architected like so:我的 Terraform 代码的架构大致如下:
module "firewall_hub" {
# This creates the Azure Firewall resource
source = "/path/to/module/a"
# attribute = value...
}
module "firewall_spoke" {
# This creates, amongst other things, firewall rule sets
source = "/path/to/module/b"
hub = module.firewall_hub
# attribute = value...
}
module "another_firewall_spoke" {
# This creates, amongst other things, firewall rule sets
source = "/path/to/module/c"
hub = module.firewall_hub
# attribute = value...
}
ie, The Azure Firewall resource is created in module.firewall_hub
, which is used as an input into module.firewall_spoke
and module.another_firewall_spoke
that create their necessary resources and inject firewall rule sets into the Firewall resource.即,Azure 防火墙资源在
module.firewall_hub
中创建,用作module.firewall_spoke
和module.another_firewall_spoke
的输入,它们创建必要的资源并将防火墙规则集注入防火墙资源。 Importantly, the rule sets are mutually exclusive between spoke modules and designed such that their priorities don't overlap.重要的是,规则集在辐射模块之间是相互排斥的,并且被设计成它们的优先级不会重叠。
When I try to deploy this code (either build or destroy), Azure throws an error:当我尝试部署此代码(构建或销毁)时,Azure 会引发错误:
Error: deleting Application Rule Collection "XXX" from Firewall "XXX (Resource Group "XXX"):.network.AzureFirewallsClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: autorest/azure: Service returned an error. Status= Code="AnotherOperationInProgress" Message="Another operation on this or dependent resource is in progress.
错误:从防火墙“XXX(资源组“XXX”)删除应用程序规则集合“XXX”:.network.AzureFirewallsClient#CreateOrUpdate:发送请求失败:StatusCode = 0 - 原始错误:autorest / azure:服务返回错误。状态= Code="AnotherOperationInProgress" Message="此资源或依赖资源上的另一个操作正在进行中。 To retrieve status of the operation use uri: https://management.azure.com/subscriptions/XXX" Details=[]
要检索操作的状态,请使用 uri: https://management.azure.com/subscriptions/XXX" Details=[]
My working hypothesis is that one cannot make multiple create/update/delete requests of firewall rule sets against the same firewall simultaneously, even if the rule sets are mutually exclusive.我的工作假设是不能同时对同一防火墙发出防火墙规则集的多个创建/更新/删除请求,即使规则集是相互排斥的。 Indeed, if you wait a minute-or-so after the failed deployment and restart it -- without changing any Terraform code or manually updating resources in Azure -- it will happily carry on without error and complete successfully.
事实上,如果您在部署失败后等待一分钟左右并重新启动它——无需更改任何 Terraform 代码或手动更新 Azure 中的资源——它会愉快地继续运行而不会出现错误并成功完成。
To test my assumption, I tried to workaround this by forcing serialisation of modules:为了测试我的假设,我试图通过强制模块序列化来解决这个问题:
module "another_firewall_spoke" {
# This creates, amongst other things, firewall rule sets
source = "/path/to/module/c"
hub = module.firewall_hub
# attribute = value...
depends_on = [module.firewall_spoke]
}
However, unfortunately, this is not possible with the way my modules are written:然而,不幸的是,我的模块的编写方式是不可能的:
Providers cannot be configured within modules using count, for_each or depends_on.
不能使用 count、for_each 或 depends_on 在模块中配置提供程序。
Short of rewriting my modules (not an option), is it possible to get around this race condition -- if that's the problem -- or would you consider it a bug with the azurerm
provider (ie, it should recognise that API error response and wait its turn, up to some timeout)?如果不重写我的模块(不是一个选项),是否有可能解决这个竞争条件——如果这是问题所在——或者你会认为它是
azurerm
提供程序的错误(即,它应该识别 API 错误响应和轮到它了,直到超时)?
(Terraform v1.1.7, azurerm
v2.96.0) (Terraform v1.1.7,
azurerm
v2.96.0)
Following @silent's tip-off to this answer , I was able to resolve the race using the method described therein.在@silent 对这个答案的提示之后,我能够使用其中描述的方法解决比赛。
Something like this:是这样的:
module "firewall_hub" {
# This creates the Azure Firewall resource
source = "/path/to/module/a"
# attribute = value...
}
module "firewall_spoke" {
# This creates, amongst other things, firewall rule sets
# Has an output "blockers" containing resources that cannot be deployed concurrently
source = "/path/to/module/b"
hub = module.firewall_hub
# attribute = value...
}
module "another_firewall_spoke" {
# This creates, amongst other things, firewall rule sets
source = "/path/to/module/c"
hub = module.firewall_hub
waits_for = module.firewall_spoke.blockers
# attribute = value...
}
So the trick is for your modules to export an output that contains a list of all the dependent resources that need to be deployed first.因此,诀窍是让您的模块导出一个 output,其中包含需要首先部署的所有依赖资源的列表。 This can then be an input to subsequent modules, that is threaded through to the actual resources that require a
depends_on
value.然后,这可以作为后续模块的输入,通过线程连接到需要
depends_on
值的实际资源。
That is, in the depths of my module, resources have:也就是说,在我的模块的深处,资源有:
resource "some_resource" "foo" {
# attribute = value...
depends_on = [var.waits_for]
}
There are two important notes to bear in mind when using this method:使用此方法时需要牢记两个重要注意事项:
The wait_for
variable in your module must have type any
;模块中的
wait_for
变量必须具有类型any
; list(any)
doesn't work, as Terraform interprets this as a homogeneous list (which it most likely won't be). list(any)
不起作用,因为 Terraform 将其解释为同类列表(很可能不会)。
Weirdly, imo, the depends_on
clause requires you to explicitly use a list literal (ie, [var.waits_for]
rather than just var.waits_for
), even if the variable you are threading through is a list.奇怪的是,imo,
depends_on
子句要求您显式使用列表文字(即[var.waits_for]
而不仅仅是var.waits_for
),即使您正在处理的变量是一个列表。 This doesn't type check in my head, but apparently Terraform is not only fine with it, but it expects it!这并没有在我的脑海中输入检查,但显然 Terraform 不仅适用于它,而且它期待它!
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.