简体   繁体   English

AWS Step Functions - 将列表传递给 Glue 的参数

[英]AWS Step Functions - pass list into Glue's argument

Within my workflow I query DynamoDB for tables whose load_fail status equals 1 .在我的工作流程中,我在 DynamoDB 中查询load_fail状态等于1的表。

If there is at least one table, Glue job needs to start with that list of tables as --source_tables argument.如果至少有一个表,则 Glue 作业需要以该表列表作为--source_tables参数开始。

Below is my entire state machine.下面是我的整个 state 机器。

{
  "Comment": "A description of my state machine",
  "StartAt": "Query",
  "States": {
    "Query": {
      "Type": "Task",
      "Next": "Choice",
      "Parameters": {
        "TableName": "source_tables_load_status",
        "KeyConditionExpression": "load_fail = :load_fail",
        "ExpressionAttributeValues": {
          ":load_fail": {
            "S": "1"
          }
        }
      },
      "Resource": "arn:aws:states:::aws-sdk:dynamodb:query",
      "ResultSelector": {
        "count.$": "$.Count",
        "startTime.$": "$$.Execution.StartTime",
        "items.$": "$.Items[*].table_name.S"
      }
    },
    "Choice": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.count",
          "NumericGreaterThanEquals": 1,
          "Next": "start_glue"
        }
      ],
      "Default": "Success"
    },
    "start_glue": {
      "Type": "Task",
      "Resource": "arn:aws:states:::glue:startJobRun",
      "Parameters": {
        "JobName": "data-moving-glue",
        "Arguments": {
          "--dynamodb_metadata_table": "metadata_table",
          "--source_tables.$": "$.items"
        }
      },
      "End": true
    },
    "Success": {
      "Type": "Succeed"
    }
  }
}

Currently I'm getting an error caused by "--source_tables.$": "$.items" .目前我收到由"--source_tables.$": "$.items"引起的错误。

Question is how to make "--source_tables":["dbo.Table_Two", "dbo.Table_Three"] working by state machine:问题是如何使 state 机器工作的"--source_tables":["dbo.Table_Two", "dbo.Table_Three"]工作:

An error occurred while executing the state 'start_glue' (entered at the event id #9). 
The Parameters '{"JobName":"data-moving-glue","Arguments":{"--dynamodb_metadata_table":"metadata_table","--source_tables":["dbo.Table_Two", "dbo.Table_Three"]}}' 
could not be used to start the Task: [The value for the field '--source_tables' must be a STRING]

I closed the result in quotes making it into a string using States.Format我用引号将结果关闭,使用States.Format将其转换为字符串

https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html

"--source_tables.$": "States.Format('{}', $.items)"

New output is:新 output 是:

"--source_tables": "[\"dbo.TableOne\",\"dbo.TableTwo\"]"

This on the other hand can be handled with a function.另一方面,这可以通过 function 来处理。

eval is used only as an example Don't use it as it can compromise your code eval仅用作示例 不要使用它,因为它可能会损害您的代码

lst = "[\"dbo.TableOne\",\"dbo.TableTwo\"]"

for t in (eval(lst)): 
    print(t)

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM