Python - Filter list of dictionaries based on if key value contains all items in another list

Question

I have a list of dictionaries that looks like this, but with around 500 items:

listOfDicts = [{'ID': 1, 'abc': {'123': 'foo'}}, ... {'ID': 7, 'abc': {'123':'foo','456': 'bar'}}]

sampleFilterList = ['123', '456']

I am trying to filter the listOfDicts for all the results where all the values in the sampleFilterList are in the key 'abc'

The result should be a list:

[{'ID': 7, 'abc': {'123':'foo','456': 'bar'}}, ...]

I tried [i for i in listOfDicts if a for a in sampleFilterList in i['abc']] , but I am getting an UnboundLocalError: local variable 'a' referenced before assignment

Answer 1

You need to move the in condition test before the for keyword in the list comprehension and also use get will be more safe, which returns a default value instead of throwing an error, if you are not sure if all the dictionaries in the list have the keyword abc :

listOfDicts = [{'ID': 1, 'abc': {'123': 'foo'}}, {'ID': 7, 'abc': {'123':'foo','456': 'bar'}}]    ​
sampleFilterList = ['123', '456']

[d for d in listOfDicts if all(s in d.get('abc', {}) for s in sampleFilterList)]
# [{'ID': 7, 'abc': {'123': 'foo', '456': 'bar'}}]

Or if use a set as of @DYZ, you can use issubset :

filterSet = set(sampleFilterList)
[d for d in listOfDicts if filterSet.issubset(d.get('abc', {}))]
# [{'ID': 7, 'abc': {'123': 'foo', '456': 'bar'}}]

Answer 2

First, convert your second list to a set for more efficient comparisons:

sampleFilterSet = set(sampleFilterList)

Now, compare the 'abc' keys for each list item to the aforesaid set:

[item for item in listOfDicts if not (sampleFilterSet - item['abc'].keys())]    
#[{'ID': 7, 'abc': {'123': 'foo', '456': 'bar'}}]

This is the fastest solution. A more Pythonic (but somewhat slower) solution is to use filter() :

list(filter(lambda item: not (sampleFilterSet - item['abc'].keys()), listOfDicts))
#[{'ID': 7, 'abc': {'123': 'foo', '456': 'bar'}}]

Answer 3

Here is a working version with nested list comprehensions. Your problem is that the a for a in... is a list comprehension, and needs to be used in constructing a new list.

[i for i in listOfDicts if [a for a in sampleFilterList if a in i['abc']] == sampleFilterList]

Answer 4

for i in zip(listOfDicts):
 a = i[0]['abc']
 print (a)

or:

for i in zip(listOfDicts):
    if 'abc' in i[0]:
        a = i
        print (a)

This is an elegant way to do it, I hope it will be useful.

Answer 5

You could try the following one-liner:

passed_the_filter = [[dictionary_entry for dictionary_entry in list_of_dicts if filter_test in dictionary_entry['abc']] for filter_test in filter]

It is a nested list comprehension that iterates through both the filter and the dictionary list. It checks if the filter is a key in the dictionary entries' "abc" value. Your problem was that you used the wrong list comprehension syntax.

---

NB You may want to note that you might not be sure that an element has a "abc" key!

---

Thank you for reading this.