[英]boto3 dynamodb batch_get_item in index not working
I have a working boto3 dynamodb query below:我在下面有一个有效的 boto3 dynamodb 查询:
from boto3.dynamodb.conditions import Key
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('gpc')
video_id = 25
response = table.query(
IndexName='skuindex',
KeyConditionExpression=Key('sku').eq('98123')
)
print(response)
Which will look inside my table 'gpc', inside the index 'skuindex' for rows where sku=98123.这将在我的表'gpc'内部,在索引'skuindex'内查找sku = 98123的行。 It returns one result correctly.
它正确返回一个结果。
I am trying to replicate this query in a batch_get_item
command, but am unable to query for the sku field, and can't figure out how to specify the index.我试图在
batch_get_item
命令中复制此查询,但无法查询 sku 字段,也无法弄清楚如何指定索引。
The following batch_get_item
code works:以下
batch_get_item
代码有效:
boto3.resource('dynamodb')
dynamo = boto3.client("dynamodb", region_name="us-east-1")
serializer = boto3.dynamodb.types.TypeSerializer().serialize
test_keys = []
test_keys.append({
"upc": serializer("8122222"),
#"sku": serializer("98123"),
"source_store": serializer("itemname_itemname")
})
test_response = dynamo.batch_get_item(
RequestItems={
"gpc": {'Keys': test_keys}
}
)
print("len(test_response['Responses']['gpc']) = ", len(test_response['Responses']['gpc']))
Is there some way I can specify inside this above to use the 'skuindex' index?有什么方法可以在上面指定使用“skuindex”索引吗? Is that even possible with the batch_get_item command?
甚至可以使用 batch_get_item 命令吗? If i uncomment the sku line, it leads to an error:
如果我取消注释 sku 行,则会导致错误:
botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the BatchGetItem operation: The provided key element does not match the schema
Because the sku field is only accessible in the 'skuindex' index apparently.因为 sku 字段显然只能在“skuindex”索引中访问。 I've tried adding index specific code to the batch_get_item query with no success.
我尝试将特定于索引的代码添加到 batch_get_item 查询中,但没有成功。
How can I tell my batch_get_item to look inside the 'skuindex' index of my 'gpc' table?如何告诉我的 batch_get_item 查看我的“gpc”表的“skuindex”索引?
Rafael Almeida's answer, saying that BatchGetItem
(and also GetItem
) cannot read using an index is correct. Rafael Almeida 的回答,说
BatchGetItem
(以及GetItem
)无法使用索引读取是正确的。 He said it is a DynamoDB limitation , but I think it's interesting to understand why using an index simply doesn't fit the GetItem
or BatchGetItem
request API, but does fit Query
:他说这是DynamoDB 的限制,但我认为理解为什么使用索引根本不适合
GetItem
或BatchGetItem
请求 API,但适合Query
是很有趣的:
One important difference between the Query
request and GetItem
/ BatchGetItem
requests is that Query
API is designed to be able to return an unbounded number of responses. Query
请求和GetItem
/ BatchGetItem
请求之间的一个重要区别是Query
API 旨在能够返回无限数量的响应。 Query
is therefore designed around the notion of paging and its ability to get back any number of items.因此,
Query
是围绕分页的概念及其取回任意数量的项目的能力而设计的。 In contrast, GetItem
will always return one item (or none), while BatchGetItem
can return more - but they will all be stuck in a single response.相比之下,
GetItem
将始终返回一个项目(或不返回),而BatchGetItem
可以返回更多 - 但它们都将卡在一个响应中。
Now, the thing is that the nature of an index is that you can never know how many items will actually match your lookup sku=98123 .现在,问题是索引的本质是您永远无法知道有多少项目实际上与您的查找sku=98123匹配。 Your application assumes it will be just one, but nothing actually prevents it from storing a million items, all having the same sku value.
您的应用程序假定它只是一个,但实际上没有什么能阻止它存储一百万个项目,所有项目都具有相同的sku值。 An item's key needs to be unique, but nothing guarantees that a non-key attribute is unique.
一个项目的键必须是唯一的,但没有什么能保证非键属性是唯一的。
So, if DynamoDB were to allow a GetItem
with an index lookup sku=98123 , it would need to somehow be able to return - potentially - either one or a million different results.因此,如果 DynamoDB 允许使用索引查找sku= 98123 的
GetItem
,它需要能够以某种方式返回(可能)一个或一百万个不同的结果。 Basically, Amazon would have needed to add support for Limit
and paging to GetItem
- but there is already a request that had exactly such support built-in, and that was Query
.基本上,亚马逊需要为
GetItem
添加对Limit
和分页的支持——但已经有一个请求内置了这种支持,那就是Query
。 This is why you need to do index queries through Query
.这就是为什么您需要通过
Query
进行索引查询的原因。
You are right - it means you can do multiple such reads in one request.你是对的——这意味着你可以在一个请求中进行多次这样的读取。 There is no
BatchQuery
request.没有
BatchQuery
请求。 But to be honest, a BatchGetItem
didn't need to exist either - users should have been able to send a stream of separate GetItem
or Query
requests on the same connection.但老实说,
BatchGetItem
也不需要存在 - 用户应该能够在同一连接上发送单独的GetItem
或Query
请求的 stream。 This could have just as efficient as today's BatchGetItem
if DynamoDB used HTTP/2 - which finally added proper support for request pipelining (and out-of-order responses).如果 DynamoDB 使用 HTTP/2,这可能与今天的
BatchGetItem
一样有效——这最终增加了对请求管道(和无序响应)的适当支持。 Maybe one day DynamoDB will support HTTP/2 (as far as I know, it still doesn't).也许有一天 DynamoDB 会支持 HTTP/2(据我所知,它仍然不支持)。
Is there some way I can specify inside this above to use the 'skuindex' index?
有什么方法可以在上面指定使用“skuindex”索引吗?
Unfortunately, you can't, and this is not a boto3
limitation but a DynamoDB limitation .不幸的是,您不能,这不是
boto3
限制,而是DynamoDB 限制。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.