[英]AWS DataPipeline via Cloudformation throws error 'type is not defined in fields'
[英]Is it possible to create an array pipeline object in AWS datapipeline via Cloudformation?
通过创建EmrCluster的API / CLI创建数据管道时,我可以使用数组结构指定多个步骤:
{ "objects" : [
{ "id" : "myEmrCluster",
"terminateAfter" : "1 hours",
"schedule" : {"ref":"theSchedule"}
"step" : ["some.jar,-param1,val1", "someOther.jar,-foo,bar"] },
{ "id" : "theSchedule", "period":"1 days" }
] }
我可以调用put-pipeline-definition
引用上面的文件来为EMR集群创建许多步骤。
现在,如果我想使用CloudFormation创建管道,则可以使用AWS::DataPipeline::Pipeline
资源类型中的PipelineObjects
属性来配置管道。 但是,管道对象只能是StringValue
或RefValue
类型。 如何创建数组管道对象字段?
这是一个相应的cloudformation模板:
"Resources" : {
"MyEMRCluster" : {
"Type" : "AWS::DataPipeline::Pipeline",
"Properties" : {
"Name" : "MyETLJobs",
"Activate" : "true",
"PipelineObjects" : [
{
"Id" : "myEmrCluster",
"Fields" : [
{ "Key" : "terminateAfter","StringValue":"1 hours" },
{ "Key" : "schedule","RefValue" : "theSchedule" },
{ "Key" : "step","StringValue" : "some.jar,-param1,val1" }
]
},
{
"Id" : "theSchedule",
"Fields" : [
{ "Key" : "period","StringValue":"1 days" }
]
}
]
}
}
}
使用上述模板, step
是一个StringValue
,等效于:
"step" : "some.jar,-param1,val1"
而不是所需配置的数组。
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-datapipeline-pipeline-pipelineobjects-fields.html仅显示StringValue
和RefValue
是有效键-是否可以通过以下方式创建步骤数组CloudFormation?
提前致谢。
嗯,我不确定在哪里可以将steps
配置为数组-文档中没有提及-而是指定要包含多个步骤,应使用多个step
条目。
{
"Id" : "myEmrCluster",
"Fields" : [
{ "Key" : "terminateAfter","StringValue":"1 hours" },
{ "Key" : "schedule","RefValue" : "theSchedule" },
{ "Key" : "step","StringValue" : "some.jar,-param1,val1" },
{ "Key" : "step","StringValue" : "someOther.jar,-foo,bar" }
]
}
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.