How do I create a nested dict from a dict by splitting the keys on a delimiter in python?

Question

I have a dict like the following:

a = { 
  'customer_name': 'bob', 
  'customer_phone': '555-1212', 
  'order_0_number': 'A33432-24',
  'order_0_date': '12/12/12',
  'order_1_number': 'asd24222',
  'order_1_date': '12/14/12'
}

and I need it split on the underscore and put into a nested dict like this:

b = {
  'customer': {
    'name': 'bob',
    'phone': '555-1212'
  },
  'order': {
    '0': {
      'date': '12/12/12',
      'number': '...' },
    '1': { ... etc.

The actual data I have is more deeply nested than this. I'm this far into it, but getting stuck on how to do it in Python:

def expand_data(field, value):

  split_field = field.split('_', 1)

  #base case, end of string
  if len(split_field) == 1:
    child_element[split_field[0] = value
    return child_element
  else:
    child_element[split_field[0]] = expand_data(split_field[1],value)
    return child_element

b = {}
for k,v in a.iteritems():
  b += expand_data(k, v) # stuck here because I can't add nested dicts together

but I'm not entirely sure this is even the right way to go about it. I haven't run this code yet, just trying to think about it right now.

Also, the dict keys may change in the future so all I can rely on is the '_' underscore being there to split on. I also won't know how deeply nested it's going to need to be.

Answer 1

A generic solution:

def nest_dict(flat_dict, sep='_'):
    """Return nested dict by splitting the keys on a delimiter.

    >>> from pprint import pprint
    >>> pprint(nest_dict({'title': 'foo', 'author_name': 'stretch',
    ... 'author_zipcode': '06901'}))
    {'author': {'name': 'stretch', 'zipcode': '06901'}, 'title': 'foo'}
    """
    tree = {}
    for key, val in flat_dict.items():
        t = tree
        prev = None
        for part in key.split(sep):
            if prev is not None:
                t = t.setdefault(prev, {})
            prev = part
        else:
            t.setdefault(prev, val)
    return tree


if __name__ == '__main__':
    import doctest
    doctest.testmod()

Answer 2

here are the steps you can use but I won't give code as you don't show us any of your code.

1. Process through your keys you can easily do it with for key in yourDico
2. Then detect your pattern customer and order to split the values to get what you want.
3. Considering what you got from the first split create dico with the new data and repeat the split if it still contains _ which seems to be the split char for you.

Answer 3

b = {}
b["customer"] = {"name": a["customer_name"], "phone": a["customer_phone"]}
for key in a.keys():
    if key.startswith("order_"):
        o, i, f = key.split("_")
        order = b.get("order", {})
        order_i = order.get(i, {})
        order_i[f] = a[key]
        b["order"] = order
        b["order"][i] = order_i

print(b)

Output:

{'customer': {'name': 'bob', 'phone': '555-1212'},
 'order': {'0': {'date': '12/12/12', 'number': 'A33432-24'},
           '1': {'date': '12/14/12', 'number': 'asd24222'}}}

Answer 4

Another generic solution with some comments:

def expand_flatdict_to_tree(dict, sep = '_', to_lower = True):
    tree = {}
    
    for src, val in dict.items():
        ref = tree

        if to_lower:
            src = src.lower()
    
        for i, part in enumerate(src.split(sep)):
            if part not in ref:
                ref[part] = {}
            if i == len(src.split(sep)) -1:
                ref[part] = val # we cannot do ref = val after loop, as assignment to the ref itself will be passed by assignment
                break
            ref = ref[part] # update nest reference
    
    return tree


flat_dict = {
    'LOGLEVEL': 'INFO',
    'DB_ENABLED': True,
    'DB_CONNECTOR_HOST': 'localhost',
    'DB_CONNECTOR_USER': 'root',
}

print (expand_flatdict_to_tree(flat_dict))

# {
#     'loglevel': 'INFO',
#     'db': {
#         'enabled': True,
#         'connector': {
#             'host': 'localhost',
#             'user': 'root'
#         }
#     }
# }