Glom spec for list of dictionaries inside a list of dictionaries

Question

I am currently using glom to parse through a JSON API response, which returns, among other things, a list of dictionaries, with a list of dictionaries inside it. The problem I'm having is getting glom to access the correct dictionary entry.

Example JSON:

{'answeredAt': '2019-08-23T21:11:04Z',
  'direction': 'Inbound',
  'disposition': 'Answered',
  'duration': 110867,
  'endedAt': '2019-08-23T21:12:55Z',
  'from': {'connectedAt': '2019-08-23T21:11:04Z',
   'departmentName': None,
   'deviceType': None,
   'disconnectedAt': '2019-08-23T21:12:55Z',
   'name': 'blah',
   'number': '1234567890',
   'number_e164': '1234567890',
   'serviceId': None,
   'userId': None},
  'initialQueueName': 'blah',
  'joinedLinkedIds': [],
  'legs': [{'departmentName': 'default',
    'deviceType': 'Unknown',
    'legType': 'Dial',
    'menuName': None,
    'menuOption': None,
    'menuPrompt': None,
    'number': '1234567890',
    'optionAction': None,
    'optionArg': None,
    'queueName': None,
    'serviceId': 327727,
    'timestamp': '2019-08-23T21:11:04Z',
    'userId': None},
   {'departmentName': 'default',
    'deviceType': 'Unknown',
    'legType': 'Answer',
    'menuName': None,
    'menuOption': None,
    'menuPrompt': None,
    'number': '1234567890',
    'optionAction': None,
    'optionArg': None,
    'queueName': None,
    'serviceId': 327727,
    'timestamp': '2019-08-23T21:11:04Z',
    'userId': None},
   {'departmentName': None,
    'deviceType': None,
    'legType': 'EnterIVR',
    'menuName': 'blah',
    'menuOption': None,
    'menuPrompt': None,
    'number': None,
    'optionAction': None,
    'optionArg': None,
    'queueName': None,
    'serviceId': None,
    'timestamp': '2019-08-23T21:11:05Z',
    'userId': None},
   {'departmentName': None,
    'deviceType': None,
    'legType': 'IVRSchedule',
    'menuName': 'Day',
    'menuOption': None,
    'menuPrompt': None,
    'number': None,
    'optionAction': None,
    'optionArg': None,
    'queueName': None,
    'serviceId': None,
    'timestamp': '2019-08-23T21:11:06Z',
    'userId': None},
   {'departmentName': None,
    'deviceType': None,
    'legType': 'EnterQueue',
    'menuName': None,
    'menuOption': None,
    'menuPrompt': None,
    'number': None,
    'optionAction': None,
    'optionArg': None,
    'queueName': 'blah',
    'serviceId': None,
    'timestamp': '2019-08-23T21:11:15Z',
    'userId': None},
   {'departmentName': None,
    'deviceType': None,
    'legType': 'Hangup',
    'menuName': None,
    'menuOption': None,
    'menuPrompt': None,
    'number': 'blah',
    'optionAction': None,
    'optionArg': None,
    'queueName': None,
    'serviceId': None,
    'timestamp': '2019-08-23T21:12:55Z',
    'userId': None}],
  'linkedId': 'some unique key',
  'startedAt': '2019-08-23T21:11:04Z',
  'to': {'connectedAt': '2019-08-23T21:11:04Z',
   'departmentName': 'default',
   'deviceType': 'Unknown',
   'disconnectedAt': '2019-08-23T21:12:55Z',
   'name': None,
   'number': '1234567890',
   'number_e164': '1234567890',
   'serviceId': 327727,
   'userId': None},
  'version': {'label': None, 'major': 4, 'minor': 2, 'point': 1}},

The information I'm trying to get at is in 'legs', where 'legType' == 'Dial' or 'EnterIVR'. I need 'number' from the 'Dial' leg, and 'menuName' from the 'EnterIVR' leg. I can get it, for instance, to list back all the different legTypes, but not the data specifically from those.

This is where I'm at currently:

with open('callstest.csv',mode='w') as calls:
    data_writer = csv.writer(calls, delimiter = ',')
    data_writer.writerow(['LinkedID','Number','Queue','Client'])
    target = response_json['calls']
    glomtemp = {}
    for item in target:
        spec = {
            'Linked ID':'linkedId',
            #this returns the number I need only in certain cases, 
            #so I need 'number' from the 'Dial' legType
            'Number': ('to', 'number') 
            'Queue': 'initialQueueName',
            'Client': #need help here, should be 'menuName' from
                      #'EnterIVR' legType
                }
        glomtemp = glom(item,spec)
        #print(glomtemp)
        data_writer.writerow([glomtemp['Linked ID'],glomtemp['Number'],glomtemp['Queue']])

Right now I can get them to fall back with Coalesce to "None", but that's not what I'm looking for.

Any suggestions on how I should spec this to get the info out of those 2 legs for 'Number' and 'Client'?

Answer 1

If I understand correctly, you want to filter out certain entries that don't fit the supported legType . You're definitely onto something with the Coalesce , and I think the key here is glom's Check specifier type , combined with the SKIP singleton . I had to tweak your current spec a bit to match the example data, but this runs:

from glom import glom, Check, Coalesce, SKIP

LEG_SPEC = {'Client': Coalesce('menuName', default=''),
            'Number': Coalesce('to.number', default=''),
            'Linked ID': 'serviceId',
            'Queue': 'queueName'}
entries_spec = ('legs', 
                [Check('legType', one_of=('Dial', 'EnterIVR'), default=SKIP)],
                [LEG_SPEC])
pprint(glom(target, entries_spec))
# prints:
# [{'Client': None, 'Linked ID': 327727, 'Number': '', 'Queue': None},
#  {'Client': 'blah', 'Linked ID': None, 'Number': '', 'Queue': None}]

Not sure if that was exactly what you were hoping to see, but the pattern is there. I think you want None s (or '' ) for those other fields because the csv you're writing is going to want to put something in those columns.

There are other ways of doing filtered iteration using glom, too. The snippets page has a short section , complete with examples.