How can I traverse a deeply nested dictionary which has lists and other dictionaries within it in Python?

Question

I'm fairly new to Python and I am trying to build up filter query as my final result with a dictionary with n-depth. Inside it could other dictionaries and lists.

This is my structure:

filters = {
   predicate: 'AND',
   filters: [
       {'property_class_id': 10, operator: 'contains', operands: ['FOO']},
       {
           predicate: 'NOT',
           filters: [{
               predicate: 'OR',
               filters: [
                   {'property_class_id': 1, operator: 'contains', operands: ['Hello']}, 
                   {'property_class_id': 2, operator: 'contains', operands: ['my search term']}
               ]
           }]
       },
       {
           predicate: 'OR',
           filters: [
               {'property_class_id': 3, operator: 'contains', operands: ['my search term']}, 
               {'property_class_id': 4, operator: 'contains', operands: ['my search term']}
            ]
       }
   ]
}

I am hoping this would translate to A + !(B OR C) + (D OR E) with Q objects.

My first problem however, is how do I traverse this dictionary going through each key value pair?

This is what I have so far but you can see the limitation once I hit a list as the for loop only accepts dictionaries.

def unpack_filter(self, filters):
    q_object = Q()
    q_list = []

    for key, value in filters.iteritems():
        if isinstance(value, list) or isinstance(value, dict):
            self.unpack_filter(value)
        else:
            print "{0} : {1}".format(key, value)

Answer 1

I've changed the structure a tiny bit to make it run. You can simply integrate the modified unpack_filter loop into your code:

base_filter = {
  'predicate': 'AND',
  'filters': [
    {'property_class_id': 10, 'operator': 'contains', 'operands': ['FOO']},
    {
      'predicate': 'NOT',
      'filters': [{
        'predicate': 'OR',
        'filters': [
          {'property_class_id': 1, 'operator': 'contains', 'operands': ['Hello']}, 
          {'property_class_id': 2, 'operator': 'contains', 'operands': ['my search term']}
        ]
      }]
    },
    {
      'predicate': 'OR',
      'filters': [
        {'property_class_id': 3, 'operator': 'contains', 'operands': ['my search term']}, 
        {'property_class_id': 4, 'operator': 'contains', 'operands': ['my search term']}
       ]
    }
  ]
}

# try to avoid using/overwriting the 'filter' name as it's a built-in function
# https://docs.python.org/2/library/functions.html#filter
# this is why I'm using names such as 'curr_filter' and 'f'

def unpack_filter(curr_filter):
  # ... do something with curr_filter['predicate'] ...
  for f in curr_filter['filters']:
    if 'filters' in f:
      unpack_filter(f)    
    else:
      for key, value in f.iteritems():
        print '{0} : {1}'.format(key, value)

unpack_filter(base_filter)

Answer 2

As stated in the previous answer you can use the 'in' operator to loop through either keys in a dictionary or items in a list. This way you can have one loop with if statements that decide how to respond. The above answer will print the keys and values of the innermost dictionaries, which might be what you want. Here is another option that will print the values of the dictionaries only when 'filters' or 'predicates' is not a key. You can then use **kwargs and pass the innermost dictionaries, if structured correctly, directly to the Q() object to create keyword arguments for the query.

def unpack_filter(curr_filter):
  for f in curr_filter:
    if f == 'filters':
      unpack_filter(curr_filter[f]) 
    elif f == 'predicate':
      print curr_filter[f]
    elif 'filters' in f or 'predicate' in f:
      unpack_filter(f)
    else:
      print f

Here is a simple example passing a dictionary as keyword arguments:

filters = {'name__icontains': 'Google'}
def f(**kwargs):
  val = kwargs.pop('name__icontains')
  print val

# the next two function calls print the same thing
f(**filters)
f(name__icontains='Google')