Dot notation to Json in python

Question

I receive data from the Loggly service in dot notation, but to put data back in, it must be in JSON.

Hence, I need to convert:

{'json.message.status.time':50, 'json.message.code.response':80, 'json.time':100}

Into:

{'message': {'code': {'response': 80}, 'status': {'time': 50}}, 'time': 100}

I have put together a function to do so, but I wonder if there is a more direct and simpler way to accomplish the same result.

def dot_to_json(a):

    # Create root for JSON tree structure
    resp = {}

    for k,v in a.items():
        # eliminate json. (if metric comes from another type, it will keep its root)
        k = re.sub(r'\bjson.\b','',k)
        if '.' in k:
            # Field has a dot
            r = resp
            s = ''
            k2 =  k.split('.')
            l = len(k2)
            count = 0
            t = {}
            for f in k2:
                count += 1
                if f not in resp.keys():
                    r[f]={}
                r = r[f]
                if count < l:
                    s += "['" + f + "']"
                else:
                    s = "resp%s" % s
                    t = eval(s)
                    # Assign value to the last branch
                    t[f] = v
        else:
            r2 = resp
            if k not in resp.keys():
                r2[k] = {}
            r2[k] = v
    return resp

Answer 1

You can turn the path into dictionary access with:

def dot_to_json(a):
    output = {}
    for key, value in a.iteritems():
        path = key.split('.')
        if path[0] == 'json':
            path = path[1:]
        target = reduce(lambda d, k: d.setdefault(k, {}), path[:-1], output)
        target[path[-1]] = value
    return output

This takes the key as a path, ignoring the first json part. With reduce() you can walk the elements of path (except for the last one) and fetch the nested dictionary with it.

Essentially you start at output and for each element in path fetch the value and use that value as the input for the next iteration. Here dict.setdefault() is used to default to a new empty dictionary each time a key doesn't yet exist. For a path ['foo', 'bar', 'baz'] this comes down to the call output.setdefault('foo', {}).setdefault('bar', {}).setdefault('baz', {}) , only more compact and supporting arbitrary length paths.

The innermost dictionary is then used to set the value with the last element of the path as the key.

Demo:

>>> def dot_to_json(a):
...     output = {}
...     for key, value in a.iteritems():
...         path = key.split('.')[1:]  # ignore the json. prefix
...         target = reduce(lambda d, k: d.setdefault(k, {}), path[:-1], output)
...         target[path[-1]] = value
...     return output
... 
>>> dot_to_json({'json.message.status.time':50, 'json.message.code.response':80, 'json.time':100}))
{'message': {'status': {'time': 50}, 'code': {'response': 80}}, 'time': 100}