Why is the JSON output of my task being escaped by AWS Step Functions?

Question

I have a Lambda function that runs in a step function.

The Lambda function returns a JSON string as output.

When I debug the function locally, I see that the JSON is valid but when I run the step function and get to the next step after my function, I can see that all my " have turned to \" and there is a " at the beginning and end of my JSON.

So a JSON object that looks like the following when I debug my function:

{"test":60,"test2":"30000","test3":"result1"}

Ends up looking like the following as the input of the step after my lambda:

"{\"test\":60,\"test2\":\"30000\",\"test3\":\"result1\"}"

Why does my valid JSON object end up being escaped?

How can I prevent this from happening?

Answer 1

The Lambda function returns a JSON string as output.

That is exactly why your JSON is being escaped - you're returning your object as a JSON string eg using JSON.stringify not as a JSON object .

The easiest way to fix this would be to just return the object & not convert the output to a JSON string. That way, it won't not be escaped & will be returned, as you expect, as an object.

However, if it must stay as a JSON string for whatever reason, you can use the States.StringToJson(...) intrinsic function to unescape the escaped JSON string using the ResultSelector property of your task.

---

So for example, if your output is:

{
    "Payload": "{\"test\":60,\"test2\":\"30000\",\"test3\":\"result1\"}",
    ...
}

To be able to unescape the output before passing it to the next task, set the ResultSelector of your task to:

"ResultSelector": {
    "Payload.$":"States.StringToJson($.Payload)"
}

Or if using the Workflow Studio, click on the task, check the Output > Transform result with ResultSelector - optional checkbox and fill in the text box with the above ResultSelector object:

---

Either way, the final result of your task definition should look like this:

{
  ...
  "States": {
    "Lambda Invoke": {
      "Type": "Task",
      ...
      "ResultSelector": {
        "Payload.$":"States.StringToJson($.Payload)"
      }
    }
  }
}

The output will then be as you expect:

{
  "Payload": {
    "test": 60,
    "test2": "30000",
    "test3": "result1"
  }
}

Answer 2

While the answer from @Ermiya Eskandary is entirely correct, you also have a few more options that you can use to your advantage with or without using ResultSelector (if its a stringified json, then you pretty much have to use ResultSelector however as that answer mentioned) property. ResultPath and OutputPath.

If you do not need the incoming event for anything else after this Lambda, then have your lambda return an Json like object (ie: if in python, return a dict)

In your State Machine Definition, then include two properties in your Lambda Task

OutputPath:"$.SomeKey",
ResultPath:"$.SomeKey"

the SomeKey has to be the same for both.

What these two lines together in the task properties is say (ResultPath) "Put the output of this lambda in the event under the key 'SomeKey'" and then (OutputPath) "only send this key on to the next Task"

If you still need the data from the Input, you can use ResultPath: alone, which will put the output of the Lambda under the key assigned and append it to the InputEvent as well.

See This documentation for more info

Answer 3

Newbie to step functions here. I noticed there are two different ways to call a lambda from step functions:

The AWS-SDK way using Resource: arn:aws:states:::aws-sdk:lambda:invoke

The "optimised" way using Resource: arn:aws:states:::lambda:invoke

I found that the optimized way does a much better job with the JSON coming back from the python Lambda whereas the AWS SDK way was an escaped mess.