将词典列表转换为唯一的词典列表

Question

I have been struggling for hours to solve the following issue without success.

I have a data structure that looks like this:

[   {   'ROOT': [   
            {   'firstElem': 'gc-3/1/0'},
            {   'SecondElem': '5.0.0.1'},
            {   'ThirdElem': '127.3.15.1'},
            {   'index': 16},
            {   'function': 'session'},
            {   'hw': '0.0.0.0'},
            {   'sw': '1.50.1.3'},
            {   'resources': [   {   'cpu-info': [   {   'cpu-peak-load': 1},
                                                     {   'cpu-avg-load': 1}]},
                                 {   'memory-total': 1},
                                 {   'memory-used': 2}]},
            ]},
    {   'ROOT': [
            {   'firstElem': 'gc-4/1/0'},
            {   'SecondElem': '5.0.0.2'},
            {   'ThirdElem': '127.3.4.1'},
            {   'index': 5},
            {   'function': 'stand'},
            {   'hw': '0.0.0.0'},
            {   'sw': '1.50.1.3'},
            {   'resources': [   {   'cpu-info': [   {   'cpu-peak-load': 1},
                                                     {   'cpu-avg-load': 1}]},
                                 {   'memory-total': 1},
                                 {   'memory-used': 2}]},
            ]}
]

I would like to traverse this data structure and combine all the dict-element with the same name and create a list instead. This is hard to explain and I have created an example structure of what I am looking for:

{
    "ROOT": [
        {
            "firstElem": "gc-3/1/0", 
            "SecondElem": "5.0.0.1", 
            "ThirdElem": "128.0.2.19", 
            "index": "13", 
            "function": "session", 
            "hw": "1.11.0.0 ", 
            "sw": "1.50.0.228 ", 
            "resources": {
                "cpu-info": {
                    "cpu-peak-load": "1", 
                    "cpu-avg-load": "1",
                }, 
                "memory-total": "1", 
                "memory-used": "2", 
            }, 
        }, 
        {
            "firstElem": "gc-4/1/0", 
            "SecondElem": "5.0.0.1", 
            "ThirdElem": "128.0.2.19", 
            "index": "13", 
            "function": "session", 
            "hw": "1.11.0.0 ", 
            "sw": "1.50.0.228 ", 
            "resources": {
                "cpu-info": {
                    "cpu-peak-load": "8", 
                    "cpu-avg-load": "1", 
                }, 
                "memory-total": "1", 
                "memory-used": "2", 
            },  
        }
    ], 
}

I am stuck with the original data structure and can not change it . Any help is appreciated. The structure provided above is just an example, since the data is received dynamically I will not know the tag-names. So please do not provide solutions that uses specific tag-name.

Answer 1

let's try this:

r = {}

def lst2dct(lst):
    return (lst if not isinstance(lst, list) else 
        {k: lst2dct(v) for e in lst for k, v in e.items()})

for e in source:
    key, val = e.items()[0]
    r.setdefault(key, []).append(lst2dct(val))

Answer 2

Here is a way:

>>> from collections import defaultdict
>>> def  combine(item):
    # Easy return if not a list: element itself
    if type(item) != type([]):
        return item
    # else call recursion
    first_ret = [(i.items()[0][0], combine(i.items()[0][1])) for i in item]

    # Here we group by same keys if any ('ROOT', for instance)
    count_keys = defaultdict(list)
    for couple in first_ret:
        count_keys[couple[0]].append(couple[1])
    return dict((k, v if len(v) > 1 else v[0]) for k, v in count_keys.iteritems())

I had to group the ROOT nodes, but it seems to be working:

>>> pprint(combine(l))
{'ROOT': [{'SecondElem': '5.0.0.1',
           'ThirdElem': '127.3.15.1',
           'firstElem': 'gc-3/1/0',
           'function': 'session',
           'hw': '0.0.0.0',
           'index': 16,
           'resources': {'cpu-info': {'cpu-avg-load': 1,
                                      'cpu-peak-load': 1},
                         'memory-total': 1,
                         'memory-used': 2},
           'sw': '1.50.1.3'},
          {'SecondElem': '5.0.0.2',
           'ThirdElem': '127.3.4.1',
           'firstElem': 'gc-4/1/0',
           'function': 'stand',
           'hw': '0.0.0.0',
           'index': 5,
           'resources': {'cpu-info': {'cpu-avg-load': 1,
                                      'cpu-peak-load': 1},
                         'memory-total': 1,
                         'memory-used': 2},
           'sw': '1.50.1.3'}]}
>>> 

Answer 3

Its a bit hacky, but you could try:

data = [   {   'ROOT': [   
            {   'firstElem': 'gc-3/1/0'},
            {   'SecondElem': '5.0.0.1'},
            {   'ThirdElem': '127.3.15.1'},
            {   'index': 16},
            {   'function': 'session'},
            {   'hw': '0.0.0.0'},
            {   'sw': '1.50.1.3'},
            {   'resources': [   {   'cpu-info': [   {   'cpu-peak-load': 1},
                                                     {   'cpu-avg-load': 1}]},
                                 {   'memory-total': 1},
                                 {   'memory-used': 2}]},
            ]},
    {   'ROOT': [
            {   'firstElem': 'gc-4/1/0'},
            {   'SecondElem': '5.0.0.2'},
            {   'ThirdElem': '127.3.4.1'},
            {   'index': 5},
            {   'function': 'stand'},
            {   'hw': '0.0.0.0'},
            {   'sw': '1.50.1.3'},
            {   'resources': [   {   'cpu-info': [   {   'cpu-peak-load': 1},
                                                     {   'cpu-avg-load': 1}]},
                                 {   'memory-total': 1},
                                 {   'memory-used': 2}]},
            ]}
]

root_list = [

        ]

final_data = {
            'ROOT' : root_list
            }

for dict in data:
    if dict['ROOT'] not in final_data['ROOT']:
        final_data['ROOT'].append(dict['ROOT'])

Answer 4

I just did some here and there stuff in the interpreter and have come up with this:

>>> i = [...] # your data
>>> rdict = {}
>>> for di in i:
 for root in di:
    if root not in rdict:
        rdict[root] = [di[root]]
    else:
        rdict[root] += [di[root]]

rdict is the type of dict you need, and it should work with multiple lists.

---

The function version of this:

def common_dict(list_of_dicts):
    i = list_of_dicts # less typing
    rdict = {}
    for di in i:
        for root in di:
            if root not in rdict:
                rdict[root] = [di[root]]
            else:
                rdict[root] += [di[root]]
return rdict