DynamoDB begins with not returning expected results

Question

I'm using NodeJS and DynamoDB. I'm never used DynamoDB before, and primary a C# developer (where this would simply just be a .Where(x => x... ) call, not sure why Amazon made it any more complicated then that). I'm trying to simply just query the table based on if an id starts with certain characters. For example, we have the year as the first 2 characters of the Id field. So something like this: 180192 , so the year is 2018. The 20 part is irrelevant, just wanted to give a human readable example. So the Id starts with either 18 or 17 and I simply want to query the db for all rows that Id starts with 18 (for example, could be 17 or whatever). I did look at the documentation and I'm not sure I fully understand it, here's what I have so far that is just returning all results and not the expected results.

   let params = {
        TableName: db.table,
        ProjectionExpression: "id,CompetitorName,code",
        KeyConditionExpression: "begins_with(id, :year)",
        ExpressionAttributeValues: {
            ':year': '18'
        }
    return db.docClient.scan(params).promise();

So as you can see, I'm thinking that this would be a begins_with call, where I look for 18 against the Id. But again, this is returning all results (as if I didn't have KeyConditionExpression at all).

Would love to know where I'm wrong here. Thanks!

UPDATE

So I guess begin_with won't work since it only works on strings and my id is not a string. As per commenters suggestion, I can use BETWEEN , which even that is not working either. I either get back all the results or Query key condition not supported error (if I use .scan , I get back all results, if I use .query I get the error)

Here is the code I'm trying.

        let params = {
        TableName: db.table,
        ProjectionExpression: "id,CompetitorName,code",
        KeyConditionExpression: "id BETWEEN :start and :end",
        ExpressionAttributeValues: {
            ':start': 18000,
            ':end': 189999
        }
    };

return db.docClient.query(params).promise();

Answer 1

It seems as if there's no actual solution for what I was originally trying to do unfortunately. Which is a huge downfall of DynamoDB. There really needs to be some way to do 'where' using the values of columns, like you can in virtually any other language. However, I have to admit, part of the problem was the way that id was structured. You shouldn't have to rely on the id to get info out of it. Anyways, I did find another column DateofFirstCapture which using with contains (all the dates are not the same format, it's a mess) and using a year 2018 or 2017 seems to be working.

Answer 2

if you want to fetch data by id, add it as the partition key. If you want to get data by part of the string, you can use "begins with" on sort key.

begins_with (a, substr)— true if the value of attribute a begins with a particular substring.

source: https://docs.amazonaws.cn/en_us/amazondynamodb/latest/developerguide/Query.html

Answer 3

begins_with and between can only be used on sort keys. For query you must always supply partition key.

So if you change your design to have unique partition key (or unique combo of partition/sort keys) and strings like 180192 as sort key you will be able to query begins_with(sortkey, ...) .