DynamoDB Query FilterExpression Multiple Condition Chaining Python

Question

I am trying to programmatically create a FilterExpression in Python for a DynamoDB query based on user provided parameter(s) for a specific Attribute (let's call it ' ATTRIBUTE1 ').

All user provided parameters which I need to filter for are in a list. For example: ['Parameter1', 'Parameter2']

Which would then take the form Attr('ATTRIBUTE1').eq(PARAMETER1)&Attr.('ATTRIBUTE1').eq(PARAMETER2)

How can I programmatically create an Attr for my FilterExpression like the above which is based on a changing number of user provided parameter(s)?

Sometimes I might have ['Parameter1'] and another time I might have ['Parameter1', 'Parameter2', 'Parameter3'] which need to turn into Attr('ATTRIBUTE1').eq('Parameter1') and Attr('ATTRIBUTE1').eq('Parameter1')&Attr('ATTRIBUTE1').eq('Parameter2')&Attr('ATTRIBUTE1').eq('Parameter3') , respectively.

I haven't been able to find a solution for this yet and would appreciate any guidance. Thanks in advance.

Answer 1

You can try the following. I run through the loop of query params to build the dynamodb conditions.

    if event.get('queryStringParameters'):
        query_params = event.get('queryStringParameters')
        for query_param, value in query_params.items():
            if query_param == 'attribute1':
                filter_expression_list.append(Attr('attribute1').gte(value))
            if query_param == 'attribute2':
                filter_expression_list.append(Attr('attribute2').eq(value))
            if query_param == 'attribute3':
                filter_expression_list.append(Attr('attribute3').eq(value))
    FilterExpression = get_filter_expression(filter_expression_list)

Update:

Can use the following code to get the filter expression. This will handle the case when there are 2 or more than 2 expressions

def get_filter_expression(filter_expression_list):
    filter_expression = None
    first = True
    for filter in filter_expression_list:
        if first:
            filter_expression = filter
            first = False
        else:
            filter_expression = filter_expression & filter
    return filter_expression

Answer 2

Here's a compact way I was able to get this working:

from functools import reduce
from boto3.dynamodb.conditions import Key, And

response = table.scan(FilterExpression=reduce(And, ([Key(k).eq(v) for k, v in filters.items()])))

For example, filters would be a dict like:

{
    'Status': 'Approved', 
    'SubmittedBy': 'JackCasey'
}

Answer 3

Combination of FilterExpression in a string form and ExpressionAttributeValues can work, consider following example:

attrs = ["attribute1", "attribute2", "attribute3"]
user_input = ["parameter1", "paramater2", "parameter3"]
ExpressionAttributeValues = {}
FilterExpression = "";
for index, input in enumerate(attrs):
    if(len(attrs)-1 == index): FilterExpression += input+"=:"+input
    else: FilterExpression += input+" = :"+input + " AND ";

for index, input in enumerate(user_input):
    AttrName = ":" + attrs[index]
    ExpressionAttributeValues[AttrName] = {
        "S" : input
    }

print(ExpressionAttributeValues) 
print(FilterExpression)

then you can use these two in your query, more on here http://boto3.readthedocs.io/en/latest/reference/services/dynamodb.html#client

Answer 4

A compact, Pythonic approach might be to manage the criteria in a dictionary, avoid the looping in favor of comprehensions, and use the string format method:

criteria = {'my_key1': 'my_value1', 'my_key2': 'my_value2'}
FilterExpression = ' AND '.join(['{0}=:{0}'.format(k) for k, v in criteria.items()])
ExpressionAttributeValues = {':{}'.format(k): {'S': v} for k, v in criteria.items()}

Answer 5

i = 0
for key, val in attr_value_dic.items():
    if i == 0:
        filterCondition = Attr(key).eq(val)
    filterConidtion = filterCondition & Attr(key).eq(val) 
    i = i + 1

response = table.query(
    KeyConditionExpression=Key(partition_key).eq(search_value)
    ,FilterExpression=filterCondition

    )  

Answer 6

My solution for chaining multiple filter expression in python.

from boto3.dynamodb.conditions import Key, Attr  
 ...
 users_table_dynamo.query(
    KeyConditionExpression=Key('pk').eq('SC#' + sc_id) & Key('sk').begins_with("GA#"),
    FilterExpression=eval("Attr('attribute1').eq('value1') & Attr('attribute2').eq('value2') & Attr('attribute3').eq('value3')"))

Also i can write something like this, for dynamic expression creation:

def constructFilterExpression(tag_ids):
expression = ""
for idx, item in enumerate(tag_ids):
    if idx == 0:
        expression += f"Attr('user_tag_id').eq('{str(item)}')"
    else:
        expression += f" & Attr('user_tag_id').eq('{str(item)}')"

return expression

so finally my query is:

filterExpression = constructFilterExpression(tag_ids)
item = users_table_dynamo.query(
    KeyConditionExpression=Key('pk').eq('SC#' + scenario_id) & Key('sk').begins_with("AT#"),
    FilterExpression=eval(filterExpression),
    ProjectionExpression="article_id")