[英]AWS Step Functions - pass list into Glue's argument
Within my workflow I query DynamoDB for tables whose load_fail
status equals 1
.在我的工作流程中,我在 DynamoDB 中查询load_fail
状态等于1
的表。
If there is at least one table, Glue job needs to start with that list of tables as --source_tables
argument.如果至少有一个表,则 Glue 作业需要以该表列表作为--source_tables
参数开始。
Below is my entire state machine.下面是我的整个 state 机器。
{
"Comment": "A description of my state machine",
"StartAt": "Query",
"States": {
"Query": {
"Type": "Task",
"Next": "Choice",
"Parameters": {
"TableName": "source_tables_load_status",
"KeyConditionExpression": "load_fail = :load_fail",
"ExpressionAttributeValues": {
":load_fail": {
"S": "1"
}
}
},
"Resource": "arn:aws:states:::aws-sdk:dynamodb:query",
"ResultSelector": {
"count.$": "$.Count",
"startTime.$": "$$.Execution.StartTime",
"items.$": "$.Items[*].table_name.S"
}
},
"Choice": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.count",
"NumericGreaterThanEquals": 1,
"Next": "start_glue"
}
],
"Default": "Success"
},
"start_glue": {
"Type": "Task",
"Resource": "arn:aws:states:::glue:startJobRun",
"Parameters": {
"JobName": "data-moving-glue",
"Arguments": {
"--dynamodb_metadata_table": "metadata_table",
"--source_tables.$": "$.items"
}
},
"End": true
},
"Success": {
"Type": "Succeed"
}
}
}
Currently I'm getting an error caused by "--source_tables.$": "$.items"
.目前我收到由"--source_tables.$": "$.items"
引起的错误。
Question is how to make "--source_tables":["dbo.Table_Two", "dbo.Table_Three"]
working by state machine:问题是如何使 state 机器工作的"--source_tables":["dbo.Table_Two", "dbo.Table_Three"]
工作:
An error occurred while executing the state 'start_glue' (entered at the event id #9).
The Parameters '{"JobName":"data-moving-glue","Arguments":{"--dynamodb_metadata_table":"metadata_table","--source_tables":["dbo.Table_Two", "dbo.Table_Three"]}}'
could not be used to start the Task: [The value for the field '--source_tables' must be a STRING]
I closed the result in quotes making it into a string using States.Format
我用引号将结果关闭,使用States.Format
将其转换为字符串
https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
"--source_tables.$": "States.Format('{}', $.items)"
New output is:新 output 是:
"--source_tables": "[\"dbo.TableOne\",\"dbo.TableTwo\"]"
This on the other hand can be handled with a function.另一方面,这可以通过 function 来处理。
eval
is used only as an example Don't use it as it can compromise your code eval
仅用作示例 不要使用它,因为它可能会损害您的代码
lst = "[\"dbo.TableOne\",\"dbo.TableTwo\"]"
for t in (eval(lst)):
print(t)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.