Pass and use input (parameters) to a lambda task from a step function

Question

I have a simple step function launching a lambda and I am looking for a way to pass parameters (event / context) to each of several consequent tasks. My step function looks like this:

{
  "Comment": "A Hello World example of the Amazon States Language using an AWS Lambda function",
  "StartAt": "HelloWorld",
  "States": {
    "HelloWorld": {
      "Type": "Task",
      "Parameters": {
        "TableName": "table_example"
      },
      "Resource": "arn:aws:lambda:ap-southeast-2:XXXXXXX:function:fields_sync",
      "End": true
    }
  }
}

In the lambda written with Python I am using a simple handler which is:

def lambda_handler(event, context):
    #...

The event and context look like this (checking the logs):

START RequestId: f58140b8-9f04-47d7-9285-510b0357b4c2 Version: $LATEST

I cannot find a way to pass parameters to this lambda and to use them in the script. Essentially, what I am trying to do is to run the same lambda passing a few different values as a parameter.

Could anyone please point me in the right direction?

Answer 1

Based on what you said: "looking for aa way to pass parameters (event / context) to each of several consequent tasks" I assumed that you want to pass non-static values to lambdas.

There are two ways to pass arguments through state machine. Via InputPath and Parameters . For differences please look here .

If you do not have any static values that you want to pass to lambda, I would do the following. Passed all parameters to step function in json format.

Input JSON for state machine

{
    "foo": 123,
    "bar": ["a", "b", "c"],
    "car": {
        "cdr": true
    }
    "TableName": "table_example"
}

In step function you are passing entire JSON explicitly to lambda using "InputPath": "$" , except for a first step where it is passed implicitly. For more about $ path syntax please look here . You also need to take care of task result, with one of multiple approaches using ResultPath . For most of cases the safest solution is to keep task result in special variable "ResultPath": "$.taskresult"

{
  "Comment": "A Hello World example of the Amazon States Language using an AWS Lambda function",
  "StartAt": "HelloWorld",
  "States": {
    "HelloWorld": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:ap-southeast-2:XXXXXXX:function:fields_sync",
      "Next": "HelloWorld2"
    },
    "HelloWorld2": {
      "Type": "Task",
      "InputPath": "$",
      "ResultPath": "$.taskresult"
      "Resource": "arn:aws:lambda:ap-southeast-2:XXXXXXX:function:fields_sync_2",
      "End": true
    }
  }
}

Which in lambda became event variable and can be access as python dictionary

def lambda_handler(event, context):
    table_example = event["TableName"]
    a = event["bar"][0]
    cdr_value = event["car"]["cdr"]
    # taskresult will not exist as event key 
    # only on lambda triggered by first state
    # in the rest of subsequent states
    # it will hold a task result of last executed state
    taskresult = event["taskresult"]

With this approach you can use multiple step functions and different lambdas and still keep both of them clean and small by moving all the logic in lambdas . Also it is easier to debug because all events variables will be the same in all lambdas, so via simple print(event) you can see all parameters needed for entire state machine and what possibly went wrong.

Answer 2

I came across this, apparently, when Resource is set to lambda ARN(for example "arn:aws:lambda:ap-southeast-2:XXXXXXX:function:fields_sync" ), you can't use the Parameters to specify the input and instead the state of the step function is passed(possibly the input to your state function, if there's no step before it).

To pass the function input via Parameters you can specify Resource as "arn:aws:states:::lambda:invoke" and the provide your FunctionName in the Parameters section:

{
    "StartAt": "HelloWorld",
    "States": {
        "HelloWorld": {
            "Type": "Task",
            "Resource": "arn:aws:states:::lambda:invoke",
            "Parameters": {
                "FunctionName": "YOUR_FUNCTION_NAME",
                "Payload": {
                    "SOMEPAYLOAD": "YOUR PAYLOAD"
                }
            },
            "End": true
        }
    }
}

You can find the documentation for invoking Lambda functions in here: https://docs.aws.amazon.com/step-functions/latest/dg/connect-lambda.html

You can also potentially use inputPath, or also have elements from your step function state function: https://docs.aws.amazon.com/step-functions/latest/dg/input-output-inputpath-params.html

Answer 3

Like Milan mentioned in his comment , you can pass on data to a Lambda function from a Step Function State.

In the Lambda function you'll need to read the event contents.

import json

def lambda_handler(event, context):
    TableName = event['TableName']

Answer 4

For some reason, specifying directly the lambda function ARN in Resource doesn't work.

The following workaround is purely with ASL definition, you just create a Pass step before with the parameters, and its output will be used as input of the next step (your step HelloWorld with the lambda call):

{
  "Comment": "A Hello World example of the Amazon States Language using an AWS Lambda function",
  "StartAt": "HelloParams",
  "States": {
    "HelloParams": {
      "Type": "Pass",
      "Parameters": {
        "TableName": "table_example"
      },
      "Next": "HelloWorld"
    },
    "HelloWorld": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:ap-southeast-2:XXXXXXX:function:fields_sync",
      "End": true
    }
  }
}

there is a workaround in another response that tells you to make the previous step using a lambda function, but it is not needed for simple cases. Context values can also be mapped, for example the current timestamp:

"HelloParams": {
  "Type": "Pass",
  "Parameters": {
    "TableName": "table_example",
    "Now": "$$.State.EnteredTime"
  },
  "Next": "HelloWorld"
}, 

Also, InputPath and ResultPath can be used to prevent overwriting the values from previous steps. For example:

{
  "Comment": "A Hello World example of the Amazon States Language using an AWS Lambda function",
  "StartAt": "HelloParams",
  "States": {
    "HelloParams": {
      "Type": "Pass",
      "Parameters": {
        "TableName": "table_example"
      },
      "ResultPath": "$.hello_prms",
      "Next": "HelloWorld"
    },
    "HelloWorld": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:ap-southeast-2:XXXXXXX:function:fields_sync",
      "InputPath": "$.hello_prms",
      "ResultPath": "$.hello_result",
      "End": true
    }
  }
}

that would save the parameters in hello_prms (so that you can reuse them in other steps) and save the result of the execution in hello_result without values from previous steps (in case you add them).