How to iterate and append key value pairs from a dictionary within a dicionary and storing values with same key into a list python

Question

I have multiple dictionaries that I would like to append and store the key/values into a list using python. I've already accomplised this, but the issue I am having is stumbling across a dictionary that has keys and values within another dictionary. For example: The code below iterates through each dictionary then appends them into one single dictionary as shown below.

[
  {
    'disabled': False,
    'id': '13456',
    'self': 'https://gooogle.com/customFieldOption/13456',
    'value': 'Food'
  },
  {
    'disabled': False,
    'id': '14532',
    'self': 'https://gooogle.com/customFieldOption/14532',
    'value': 'Dessert'
  }
]

Then, I iterate through the above dictionaries that carries the same key value pairs and appends each one into a list. I am storing the same key value pairs into a list. The below shows the output.

defaultdict(<class'list'>,
{
  'disabled': [
    False,
    False
  ],
  'id': [
    '13456',
    '14532'
  ],
  'self': [
    'https://gooogle.com/customFieldOption/13456',
    'https://gooogle.com/customFieldOption/14532'
  ],
  'value': [
    'Food',
    'Dessert'
  ]
})

Now the problem that I am having issues with is coming across a dictionary within dictionary that also has key value pairs. Like below:

[
  {
    'fields': {
      'issuetype': {
        'id': '23434',
        'name': 'Food',
        'self': 'www.google.com/23434',
      },
      'priority': {
        'id': '12345',
        'name': 'Dessert',
        'self': 'www.google.com/12345'
      },
      'status': {
        'id': '15432',
        'name': 'To Do',
        'self': 'www.google.com/status/15432',
        'statusCategory': {
          'id': 2,
          'key': 'new',
          'name': 'To Do',
          'self': 'www.google.com/statuscategory/2'
        }
      },
      'summary': 'PL-Bus-ICD-1882'
    },
    'id': '12356',
    'key': 'eed-1234',
    'self': 'www.google.com/issue/12356'
  },
  {
    'fields': {
      'issuetype': {
        'id': '11200',
        'name': 'House',
        'self': 'www.google.com/issuetype/11200',
      },
      'priority': {
        'id': '123400',
        'name': 'Nonessential',
        'self': 'www.google.com/priority/123400'
      },
      'status': {
        'id': '10035',
        'name': 'To Do',
        'self': 'www.google.com/status/10035',
        'statusCategory': {
          'id': 2,
          'key': 'new',
          'name': 'To Do',
          'self': 'www.google.com/statuscategory/2'
        }
      },
      'summary': 'PL-Bus-ICD-1885'
    },
    'id': '34262',
    'key': 'ekr-1254',
    'self': 'www.google.com/issue/34262'
  }
]

---

elif(key == 'customfield_12604'):
                if(oIssues[key] == None):
                    values = {'name': None }
                    issueRequirementDictNone = dict(values)
                    iterateDictIssues(issueRequirementDictNone, listInner)
                else:
                    newDictionaryValuesRequirement = []
                    for requirementID in oIssues['customfield_12604']:
                        if(len(oIssues[key]) == 1):
                            values = requirementID
                            issueRequirementID = dict(values)
                            iterateDictIssues(issueRequirementID, listInner)
                        else:
                            values = requirementID
                            newDictionaryValuesRequirement.append(values)
                            if(len(newDictionaryValuesRequirement) == len(oIssues[key])):
                                print(newDictionaryValuesRequirement)
                                d = defaultdict(list)
                                for dictionary in newDictionaryValuesRequirement:
                                    for k, v in dictionary.items():
                                        d[k].append(v)
                                values = d
                                print(values)             
                                issueRequirementDictID = dict(values)
                                iterateDictIssues(issueRequirementDictID, listInner)

My code seems to not iterate through the additionally dictionary keys. I would like to accomplish the same result as I did from my ouput above with append through the issuetype, priority and status key/value pairs into a single list showing a single dictionary. When I run the same code using the dictionary above it only stores the key/value pairs below because those are inside of the fields dictionary. I need to iterate through the other dictionaries Im assuming. Thank you.

Answer 1

A recursive approach works well here.

You can iterate over data and call a recursive function merge which takes a dictionary d and merges it into another dictionary merged on matching keys. Each dict value in d results in another recursive call, while other values get appended to the value of the corresponding key.

def merge(d, merged):
    for k, v in d.items():
        if isinstance(v, dict):
            merge(v, merged.setdefault(k, {}))
        else:
            merged.setdefault(k, []).append(v)

out = {}
for d in data:
    merge(d, out)

Output:

{'fields': {'issuetype': {'id': ['23434', '11200'],
   'name': ['Food', 'House'],
   'self': ['www.google.com/23434', 'www.google.com/issuetype/11200']},
  'priority': {'id': ['12345', '123400'],
   'name': ['Dessert', 'Nonessential'],
   'self': ['www.google.com/12345', 'www.google.com/priority/123400']},
  'status': {'id': ['15432', '10035'],
   'name': ['To Do', 'To Do'],
   'self': ['www.google.com/status/15432', 'www.google.com/status/10035'],
   'statusCategory': {'id': [2, 2],
    'key': ['new', 'new'],
    'name': ['To Do', 'To Do'],
    'self': ['www.google.com/statuscategory/2',
     'www.google.com/statuscategory/2']}},
  'summary': ['PL-Bus-ICD-1882', 'PL-Bus-ICD-1885']},
 'id': ['12356', '34262'],
 'key': ['eed-1234', 'ekr-1254'],
 'self': ['www.google.com/issue/12356', 'www.google.com/issue/34262']}