[英]How to send multiple batch messages from a list to an SQS queue using Boto3 and Python
I have a list of integers (integerList) that I'd like to pass into an SQS queue where each message into the queue is an integer from the list.我有一个整数列表 (integerList),我想将其传递到 SQS 队列中,其中进入队列的每条消息都是列表中的 integer。
I can do this one message at a time with the send_message()
command, and the code for that is below.我可以使用
send_message()
命令一次发送一条消息,代码如下。
import boto3
sqsResource = boto3.resource('sqs')
def write_sqs(integerList):
queue = sqsResource.get_queue_by_name(QueueName=NAMEOFQUEUEHERE)
for i in integerList:
response = queue.send_message(MessageBody=str(i),
MessageGroupId='TESTING')
However, I'd like to speed up the function and send the messages in batches.不过,我想加快function的速度,批量发送消息。 Currently, AWS SQS allows batching up to 10 messages at a time with the
send_messages()
command, but I'm not sure how to build the Entries=
attribute for the batch send.目前,AWS SQS 允许使用
send_messages()
命令一次最多批处理 10 条消息,但我不确定如何为批量发送构建Entries=
属性。 I'm breaking down the integerList into smaller lists of 10 using chunks = [integerList[x:x+10] for x in range(0, len(integerList), 10)]
, but the next steps are unclear.我使用
chunks = [integerList[x:x+10] for x in range(0, len(integerList), 10)]
将 integerList 分解为更小的 10 列表,但接下来的步骤尚不清楚。
According to the documentation Entries
is a list of messages. 根据文档,
Entries
是一个消息列表。
For each entry in Entries
the parameters type are detailed at the link. 对于每个条目
Entries
的参数类型是在链路详细。
import boto3
sqsResource = boto3.resource('sqs')
def write_sqs(integerList):
queue = sqsResource.get_queue_by_name(QueueName=NAMEOFQUEUEHERE)
entries = []
for i in integerList:
entry = {
'Id': 'id%s' % str(integerList[i]),
'MessageBody': str(integerList[i])
}
entries.append(entry)
response = queue.send_messages(entries)
Building on the answer from @liorko and some trial and error, this seems to work and is much faster than the 1-by-1 method I was using before. 基于@liorko的答案和一些反复试验,这似乎有效,并且比我之前使用的1-by-1方法快得多。
import boto3
sqsResource = boto3.resource('sqs')
def write_sqs(integerList):
queue = sqsResource.get_queue_by_name(QueueName=NAMEOFQUEUEHERE)
maxBatchSize = 10 #current maximum allowed
chunks = [integerList[x:x+maxBatchSize] for x in range(0, len(integerList), maxBatchSize)]
for chunk in chunks:
entries = []
for x in chunk:
entry = {'Id': str(x),
'MessageBody': str(x),
'MessageGroupId': 'ANYTHINGYOUWANT'}
entries.append(entry)
response = queue.send_messages(Entries=entries)
According to the new version of boto3, the method to use is send_message_batch
, here is the updated code:根据boto3的新版本,使用的方法是
send_message_batch
,这里是更新的代码:
import boto3
sqs_client = boto3.client("sqs", region_name="eu-west-1")
entries = []
for i in integer_list:
entry = {
'Id': 'id%s' % str(integerList[i]),
'MessageBody': str(integerList[i])
}
entries.append(entry)
response = queue.send_message_batch(Entries=entries, QueueUrl="https://sqs.eu-west-1.amazonaws.com/000000000000/my-queue-name")
Source: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sqs.html#SQS.Client.send_message_batch资料来源: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sqs.html#SQS.Client.send_message_batch
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.