AWS Step Function - Dynamic parallelism MaxConcurrency Field

Question

We are using Step functions dynamic parallelism with Map state to achieve concurrency. Is it possible to pass the values to "MaxConcurrency" field from its upstream task(lambda or read from file ) in map state stepfunctions.

Current code:

"Type": "Map",
"InputPath": "$.detail",
"ItemsPath": "$.shipped",
"MaxConcurrency": 3,
"ResultPath": "$.detail.shipped",

Expectation(pass input to MaxConcurrency from lambda task or read file task ):

 "Type": "Map",
 "InputPath": "$.detail",
 "ItemsPath": "$.shipped",
 "MaxConcurrency": "$.input",
 "ResultPath": "$.detail.shipped"

Getting error as it supports only integer.

Answer 1

As you experienced, Map's maxConcurrency expects a number , but the state machine language uses strings to pass variables dynamically.

A workaround is to add a Choice state that branches to discrete map states based on an input variable. Each branch's Map has a different maxConcurrency , but is otherwise identical. Add whichever discrete choices you need. Choice also accepts a default case to catch unmatched choice input.

// execution input
{
  "concurrency": 5,
  "jobs": [ { "jobId": 1 }, { "jobId": 2 }, { "jobId": 3 }, { "jobId": 4}]
}

// state machine definition (partial)
"States": {
  "Max-Concurrency-Choice": {
    "Type": "Choice",
    "Choices": [
      {
        "Variable": "$.concurrency",
        "NumericEquals": 5,
        "Next": "MapState-MaxConcurrency-5"
      },
      {
        "Variable": "$.concurrency",
        "NumericEquals": 10,
        "Next": "MapState-MaxConcurrency-10"
      }
    ],
    "Default": "MapState-MaxConcurrency-1"
  },