Access JSON nested arrays in Python

Question

I'm new to Python and I'm trying to process something and having no luck finding the answer or if it's already been asked. I'm making a call to an API and receiving some data back as JSON. I'm stripping out certain bits that I don't need with the keys being stripped out and only the values remaining which wouldn't be a problem but I can't get into them as the keys I want to access are nested in an array.

I've been accessing the data and can get up to json.dumps(payload['output']['generic']) but I can't seem to find any information online as to how I can access these last values only.

Apologies in advance if this question already exists.

 { "output": { "generic": [ { "response_type": "text", "text": "hi" } ], "intents": [ { "intent": "CollectionDate", "confidence": 0.8478035449981689 } ], "entities": [ { "entity": "Payslip", "location": [ 19, 26 ], "value": "When is my collection date", "confidence": 1 } ] }, "context": { "global": { "system": { "turn_count": 10 } }, "skills": { "main skill": { "user_defined": { "DemoContext": "Hi," }: "system": {} } } } }

To clarify:

I want to access the "text", "intent" and "confidence"

at the moment I'm printing the value posted and then the responses for the sections I want like the below.

print(x)
print(json.dumps(payload['output']['generic']))
print(json.dumps(payload['output']['intents']))

Answer 1

Use following code to convert the json to a dict first:

json_data = json.loads(str(yourData))

After that, in your case, the outermost key is "output", and it is another dict, so just use json_data['output'] to access the content inside.

For other keys inside of the "output", like "generic", you can see it is an array with the [] brackets. use json_data['output'][index] first to get the content inside, then use the same method you access a dict to access the content inside of keys like this.

Answer 2

They key here is that the Traceback error indicates an issue with indexing a "List"

This is because a "List" type is a valid JSON type, and generic contains a list of length 1, with a dict inside!

>>> payload['output']['generic']['text']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: list indices must be integers or slices, not str
>>> type(payload['output']['generic'])
<class 'list'>
>>> len(payload['output']['generic'])
1
>>> payload['output']['generic'][0]
{'response_type': 'text', 'text': 'hi'}
>>> type(payload['output']['generic'][0])
<class 'dict'>
>>> payload['output']['generic'][0]['text']
'hi'
>>>

So, given your expected input JSON format, you will need to know how to index in to pull each required data point.

There are a few packages, glom is one, that will help you deal with missing values from API generated JSON.