SQS 的 AWS put-bucket-notification-configuration 抛出“无法验证以下目标配置”

Question

我想将 s3:CreateObject:* 事件发送到 SQS 队列。 但是设置通知配置导致A client error (InvalidArgument) occurred when calling the PutBucketNotificationConfiguration operation: Unable to validate the following destination configurations

这是我创建存储桶的方式：

aws s3api create-bucket --profile default --bucket my-bucket --create-bucket-configuration LocationConstraint=eu-west-1

这就是我创建 SQS 队列的方式

aws sqs create-queue --profile default --queue-name my-queue --attributes file://attributes.json

使用 attributes.json 文件

{
  "DelaySeconds":"0",
  "MessageRetentionPeriod":"3600",
  "Policy":"{\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":\"*\",\"Action\":[\"sqs:SendMessage\",\"sqs:ReceiveMessage\"],\"Condition\":{\"ArnLike\": {\"aws:SourceArn\": \"arn:aws:s3:*:*:my-bucket\"}}}]}"
}

最后尝试设置抛出我上面列出的错误消息的通知：

aws s3api put-bucket-notification-configuration --profile default --bucket my-bucket --notification-configuration file://notification.json`

使用 notification.json 文件

{
  "TopicConfigurations": [
  ],
  "QueueConfigurations": [
    {
      "QueueArn": "arn:aws:sqs:eu-west-1:123456789012:my-queue",
      "Events": [
        "s3:ObjectCreated:*"
      ],
      "Filter": {
        "Key": {
          "FilterRules": [
            {
              "Name": "prefix",
              "Value": "my-filter"
            }
          ]
        }
      }
    }
  ],
  "LambdaFunctionConfigurations": [
  ]
}

我真的不知道错误可能在哪里。 谢谢你的帮助！

Answer 1

看起来您的 SQS 策略不起作用。 尝试将Id添加到您的策略中，并将Resource添加到您的声明中。 像这样的东西：

{ "DelaySeconds":"0", "MessageRetentionPeriod":"3600", "Policy":"{\\"Id\\":\\"someid\\",\\"Statement\\":[{\\"Effect\\":\\"Allow\\",\\"Resource\\": \\"arn:aws:sqs:eu-west-1:123456789012:my-queue\\",\\"Principal\\":\\"*\\",\\"Action\\":[\\"sqs:SendMessage\\",\\"sqs:ReceiveMessage\\"],\\"Condition\\":{\\"ArnLike\\": {\\"aws:SourceArn\\": \\"arn:aws:s3:*:*:my-bucket\\"}}}]}" }

以下是更多信息：

http://docs.aws.amazon.com/AmazonS3/latest/dev/ways-to-add-notification-config-to-bucket.html#step1-create-sqs-queue-for-notification

此外，从命令行调用 API 时，您可以使用 --debug 参数。 您会看到完整的错误消息：

aws --debug s3api ...

Answer 2

我有一个有效的脚本。 我把它贴在这里给任何可能对此感到困惑的人:-)

#!/usr/bin/env python

import boto3
import json

bucket_name='spike-bucket-000'
queue_name='spike_queue_000'
region='eu-west-1'

s3 = boto3.client('s3', region)
sqs = boto3.client('sqs', region)

def check_if_bucket_exists(name):
    s3.head_bucket(Bucket=bucket_name)


try:
    check_if_bucket_exists(bucket_name)
    print('Bucket {} exists'.format(bucket_name))
except botocore.exceptions.ClientError:
    print('Creating bucket {}'.format(bucket_name))
    s3.create_bucket(Bucket=bucket_name, CreateBucketConfiguration={'LocationConstraint': region})

print('Ensuring queue {} exists'.format(queue_name))

response = sqs.create_queue(QueueName=queue_name)
queue_url = response['QueueUrl']
response = sqs.get_queue_attributes(QueueUrl=queue_url, AttributeNames=['QueueArn'])
queue_arn = response['Attributes']['QueueArn']

print('Granting bucket permission to post messages to queue')

queue_policy={
         "Version": "2008-10-17",
         "Id": "example-ID",
         "Statement": [
          {
           "Sid": "example-statement-ID",
           "Effect": "Allow",
           "Principal": {
            "AWS":"*"
           },
           "Action": [
            "SQS:SendMessage"
           ],
           "Resource": queue_arn,
           "Condition": {
              "ArnLike": {
              "aws:SourceArn": "arn:aws:s3:*:*:" + bucket_name
            }
           }
          }
         ]
        }

sqs.set_queue_attributes(QueueUrl=queue_url, Attributes={'Policy': json.dumps(queue_policy)})

print('Configuring bucket to notify object creation to queue')

response = s3.put_bucket_notification_configuration(
    Bucket=bucket_name,
    NotificationConfiguration={
        'QueueConfigurations': [
            {
                'Id': 'Notify-ObjectCreated-To-Queue',
                'QueueArn': queue_arn,
                'Events': [
                    's3:ObjectCreated:*',
                ]
#               ,
#                 'Filter': {
#                     'Key': {
#                         'FilterRules': [
#                             {
#                                 'Name': 'prefix'|'suffix',
#                                 'Value': 'string'
#                             },
#                         ]
#                     }
#}
            },
        ]
    }
)

Answer 3

感谢 Lubo Sach 的回答，我可以在 SQS 中使用此政策来完成这项工作：

{
  "Version": "2008-10-17",
  "Id": "27097a52-cae3-49fe-84ce-0020893e394c",
  "Statement": [
    {
      "Sid": "__owner_statement",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::9970XXXX4660:root"
      },
      "Action": "SQS:*",
      "Resource": "arn:aws:sqs:us-east-1:9970XXXX4660:bucketname"
    },
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": [
        "sqs:SendMessage",
        "sqs:ReceiveMessage"
      ],
      "Resource": "arn:aws:sqs:us-east-1:9970XXXX4660:bucketname",
      "Condition": {
        "ArnLike": {
          "aws:SourceArn": "arn:aws:s3:*:*:bucketname"
        }
      }
    }
  ]
}

Answer 4

需要有对应的 Lambda:Invoke 权限； 确保 Lambda 拥有对存储桶的权限（如果将某些 Invoke:Permissions 授予存储桶和密钥，则会中断）