NoSql and query by list

Question

I have a DynamoDB database with roughly 100.000 items in. The items are mostly a flat, but they do contain a list of categories.

{
  "name": "Protine shaker",
  "categories": [
     "Sport",
     "Kitchen"
  ];
}

I want to be able to find all items that for an example is a member of the category "Sport", but as it is a list can i not index it. My only option with a pure DyanmoDB solution would therefore be to scan the whole table, which is something that can be expensive if done often.

I where thinking that i could have a SQL database besides the DynamoDB that just contained the category mapping, or have another DynamoDB tales with an object for each category containing a list of items in that category.

What is the normal/common solutions to such a problem when working with DynamoDB? I am guesses that this must be a common problem to have.

Answer 1

You can integrate DynamoDB with ElasticSearch which can provide more options to query for data that is not supported by DynamoDB out of the box. Just create indices in ElasticSearch and perform your queries there.

Where to get started: https://aws.amazon.com/about-aws/whats-new/2015/08/amazon-dynamodb-elasticsearch-integration/

Having a SQL database is not a good solution because you will have to keep the information in sync in two different data-stores by yourself. Looks like an operation pain for maintenance.

Having multiple DynamoDB tables per Category is not scalable and even here you would have multiple DynamoDB tables to update in a "transaction" which is possible but comes with degraded performance or you could have AWS lambda write to relevant Category table based on DynamoDB update stream. But still it is better to avoid for the sake of scalability. You would have to create a new table every time a new Category is introduced.