Adding SQS redrive policy using AWS CLI command

Question

I am trying to set redrive policy for SQS using the AWS CLI Command below, but seeing an error related to redrive JSON. Can you please let me know how I can fix this?

redrive_policy="{\"RedrivePolicy\":{\"deadLetterTargetArn\":\"$dlq_arn\",\"maxReceiveCount\":\"15\"}}"

AWS CLI COMMAND

aws sqs set-queue-attributes --queue-url https://queue.amazonaws.com/12345678/test-queue --attributes $redrive_policy --region=us-east-1

Error Message

Parameter validation failed: Invalid type for parameter Attributes.RedrivePolicy, value: OrderedDict([(u'deadLetterTargetArn', u'arn:aws:sqs:us-east-1:12345678:dlq'), (u'maxReceiveCount', u'15')]), type: , valid types:

Answer 1

Have you tried just creating the JSON in a separate file and passing it as an argument to your AWS CLI command? I find it's difficult to get all of the escaping correct when passing the JSON as a parameter. So you'd basically do it as the example shows in the AWS documentation:

https://docs.aws.amazon.com/cli/latest/reference/sqs/set-queue-attributes.html#examples

1. So first you'd create a new file called "set-queue-attributes.json" like so:

    {
      "DelaySeconds": "10",
      "MaximumMessageSize": "131072",
      "MessageRetentionPeriod": "259200",
      "ReceiveMessageWaitTimeSeconds": "20",
      "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:us-east-1:80398EXAMPLE:MyDeadLetterQueue\",\"maxReceiveCount\":\"1000\"}",
      "VisibilityTimeout": "60"
    }

2. Then run the command like this:

aws sqs set-queue-attributes --queue-url https://sqs.us-east-1.amazonaws.com/80398EXAMPLE/MyNewQueue --attributes file://set-queue-attributes.json --region=us-east-1

Answer 2

Three Methods to achieve this:

Note: The solutions also work on any other AWS CLI commands that require a stringified JSON

1. Using the Command-line JSON processor jq (Recommended)

This method is recommended because of many reasons:

• I've found jq a handy tool to use when working with AWS CLI as the need to stringify JSON comes up quite frequently.

Install for Ubuntu: sudo apt install jq

Basic Options:

• jq -R : Returns the stringified JSON
• jq -c : Eliminates spacing and newline characters

The benefit is that you can write JSON as JSON and Pipe the result into the jq -R command.

Method 1:

aws sqs set-queue-attributes \
    --queue-url "https://sqs.ap-south-1.amazonaws.com/IAMEXAMPLE12345678/ExampleQueue" \
    --attributes RedrivePolicy=$(echo '{"maxReceiveCount":500,"deadLetterTargetArn":"arn:aws:sqs:ap-south-1:IAMEXAMPLE12345678:ExampleDeadLetterQueue"}' | jq -R)

OR if you have a sqs-redrive-policy.json file:

Method 2:

In sqs-redrive-policy.json ,

{
  "maxReceiveCount": 500,
  "deadLetterTargetArn": "arn:aws:sqs:ap-south-1:IAMEXAMPLE12345678:ExampleDeadLetterQueue"
}

Run in Command Line:

   aws sqs set-queue-attributes \
    --queue-url "https://sqs.ap-south-1.amazonaws.com/IAMEXAMPLE12345678/ExampleQueue" \ 
    --attributes RedrivePolicy=$(cat ~/path/to/file/sqs-redrive-policy.json | jq -c | jq -R)

• As you can see the second benefit is that you can isolately modify only the --redrive-policy without having to touch any of the other attributes.

Common Confusion : A confusion is the name set-queue-attributes (it would be better named put-queue-attributes ). as it doesn't overwrite all attributes but only overwrites the attributes mentioned with the command. So if you already set a Policy attribute earlier during create-queue , this will not overwrite the Policy to null. In other words, this is safe to use.

---

2. Using a stringified JSON

This is a pain to be honest, and I avoid this.

aws sqs set-queue-attributes \
--queue-url "https://sqs.us-east-1.amazonaws.com/IAMEXAMPLE12345678/ExampleQueue" \
--attributes '{
    "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:ap-south-1:IAMEXAMPLE12345678:ExampleDeadLetterQueue\",\"maxReceiveCount\":\"500\"}",
}'

3. Use a filePathURL to the JSON file for attributes.json NOT sqs-redrive-policy.json

This is my last preference.

Reason :

• This means setting all the attributes specified in the attributes.json file again at a single go.
• Doesn't escape the pain of writing stringified JSON as text.

In attributes.json ,

{
  "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:ap-south-1:IAMEXAMPLE12345678:ExampleDeadLetterQueue\", \"maxReceiveCount\":\"5\"}"
}

Run in command line :

aws sqs set-queue-attributes \
    --queue-url "https://sqs.ap-south-1.amazonaws.com/IAMEXAMPLE12345678/ExampleQueue" \
    --attributes file:///home/yourusername/path/to/file/attributes.json

---

Answer 3

if you want to run in the same command you can use this example:

aws sqs set-queue-attributes \
--queue-url https://sqs.us-east-1.amazonaws.com/80398EXAMPLE/MyNewQueue \
--attributes '{
    "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:us-east-1:80398EXAMPLE:MyDeadLetterQueue\",\"maxReceiveCount\":\"1000\"}",
    "MessageRetentionPeriod": "259200",
    "VisibilityTimeout": "90"
}'