How to extract JSON from a nested JSON file?

Question

I am calling an API and getting a response like the below.

{
    "status": 200,
    "errmsg": "OK",
    "data": {
        "total": 12,
        "items": [{
                "id": 11,
                "name": "BBC",
                "priority": 4,
                "levelStr": "All",
                "escalatingChainId": 3,
                "escalatingChain": {
                    "inAlerting": false,
                    "throttlingAlerts": 20,
                    "enableThrottling": true,
                    "name": "Example123",
                    "destination": [],
                    "description": "",
                    "ccdestination": [],
                    "id": 3,
                    "throttlingPeriod": 10
                }
            },
            {
                "id": 21,
                "name": "CNBC",
                "priority": 4,
                "levelStr": "All",
                "escalatingChainId": 3,
                "escalatingChain": {
                    "inAlerting": false,
                    "throttlingAlerts": 20,
                    "enableThrottling": true,
                    "name": "Example456",
                    "destination": [],
                    "description": "",
                    "ccdestination": [],
                    "id": 3,
                    "throttlingPeriod": 10
                }
            }
        ]
    }
}

I need to clean-up this JSON a bit and produce a simple JSON like below where escalatingChainName is the name in the escalatingChain list so that I can write this into a CSV file.

{
    "items": [{
            "id": 11,
            "name": "BBC",
            "priority": 4,
            "levelStr": "All",
            "escalatingChainId": 3,
            "escalatingChainName": "Example123"
        },
        {
            "id": 21,
            "name": "CNBC",
            "priority": 4,
            "levelStr": "All",
            "escalatingChainId": 3,
            "escalatingChainName": "Example456"
        }
    ]
}

Is there a JSON function that I can use to copy only the necessary key-value or nested key-values to a new JSON object?

With the below code, I am able to get the details list.

json_response = response.json()
items = json_response['data']
details = items['items']

I can print individual list items using

for x in details:
    print(x)

How do I take it from here to pull only the necessary fields like id, name, priority and the name from escalatingchain to create a new list or JSON?

Answer 1

You can try:

data = {
    "status": 200,
    "errmsg": "OK",
    "data": {
        "total": 12,
        "items": [{
                "id": 11,
                "name": "BBC",
                "priority": 4,
                "levelStr": "All",
                "escalatingChainId": 3,
                "escalatingChain": {
                    "inAlerting": False,
                    "throttlingAlerts": 20,
                    "enableThrottling": True,
                    "name": "Example123",
                    "destination": [],
                    "description": "",
                    "ccdestination": [],
                    "id": 3,
                    "throttlingPeriod": 10
                }
            },
            {
                "id": 21,
                "name": "CNBC",
                "priority": 4,
                "levelStr": "All",
                "escalatingChainId": 3,
                "escalatingChain": {
                    "inAlerting": False,
                    "throttlingAlerts": 20,
                    "enableThrottling": True,
                    "name": "Example456",
                    "destination": [],
                    "description": "",
                    "ccdestination": [],
                    "id": 3,
                    "throttlingPeriod": 10
                }
            }
        ]
    }
}

for single_item in data["data"]["items"]:
    print(single_item["id"])
    print(single_item["name"])
    print(single_item["priority"])
    print(single_item["levelStr"])
    print(single_item["escalatingChain"]["inAlerting"])
    # and so on

Answer 2

Two ways of approaching this depending on whether your dealing with a variable or .json file using python list and dictionary comprehension:

1. Where data variable of type dictionary (nested) already defined:

    # keys you want to_keep = ['id', 'name', 'priority', 'levelStr', 'escalatingChainId', 'escalatingChainName'] new_data = [{k:v for k,v in low_dict.items() if k in to_keep} for low_dict in data['data']['items']] # where item is dictionary at lowest level escalations = [{v+'Name':k[v]['name']} for k in data['data']['items'] for v in k if type(k[v])==dict] # merge both lists of python dictionaries to produce flattened list of dictionaries new_data = [{**new,**escl} for new,escl in zip(new_data,escalations)]

2. Or (and since your refer json package) if you have save the response to as a .json file:

    import json with open('response.json', 'r') as handl: data = json.load(handl) to_keep = ['id', 'name', 'priority', 'levelStr', 'escalatingChainId', 'escalatingChainName'] new_data = [{k:v for k,v in low_dict.items() if k in to_keep} for low_dict in data['data']['items']] escalations = [{v+'Name':k[v]['name']} for k in data['data']['items'] for v in k if type(k[v])==dict] new_data = [{**new,**escl} for new,escl in zip(new_data,escalations)]

Both produce output:

[{'id': 11,
  'name': 'BBC',
  'priority': 4,
  'levelStr': 'All',
  'escalatingChainId': 3,
  'escalatingChainName': 'Example123'},
 {'id': 21,
  'name': 'CNBC',
  'priority': 4,
  'levelStr': 'All',
  'escalatingChainId': 3,
  'escalatingChainName': 'Example456'}]

Answer 3

There is no existing function that will do what you want, so you'll need to write one. Fortunately that's not too hard in this case — basically you just create a list of new items by extracting the pieces of data you want from the existing ones.

import json

json_response = """\
    {
        "status": 200,
        "errmsg": "OK",
        "data": {
            "total": 12,
            "items": [{
                    "id": 11,
                    "name": "BBC",
                    "priority": 4,
                    "levelStr": "All",
                    "escalatingChainId": 3,
                    "escalatingChain": {
                        "inAlerting": false,
                        "throttlingAlerts": 20,
                        "enableThrottling": true,
                        "name": "Example123",
                        "destination": [],
                        "description": "",
                        "ccdestination": [],
                        "id": 3,
                        "throttlingPeriod": 10
                    }
                },
                {
                    "id": 21,
                    "name": "CNBC",
                    "priority": 4,
                    "levelStr": "All",
                    "escalatingChainId": 3,
                    "escalatingChain": {
                        "inAlerting": false,
                        "throttlingAlerts": 20,
                        "enableThrottling": true,
                        "name": "Example456",
                        "destination": [],
                        "description": "",
                        "ccdestination": [],
                        "id": 3,
                        "throttlingPeriod": 10
                    }
                }
            ]
        }
    }
"""


response = json.loads(json_response)

cleaned = []
for item in response['data']['items']:
    cleaned.append({'id': item['id'],
                    'name': item['name'],
                    'priority': item['priority'],
                    'levelStr': item['levelStr'],
                    'escalatingChainId': item['escalatingChainId'],
                    'escalatingChainName': item['escalatingChain']['name']})

print('cleaned:')
print(json.dumps(cleaned, indent=4))