Navigating Multi-Dimensional JSON arrays in Python

Question

I'm trying to figure out how to query a JSON array in Python. Could someone show me how to do a simple search and print through a fairly complex array please?

The example I'm using is here: http://eu.battle.net/api/wow/realm/status

I'd like to see, for example, how to find the 'Silvermoon' server, and print say its 'population', then the 'controlling-faction' within the 'Wintergrasp' array.

The array snippet currently looks like this:

{"type":"pve","population":"high","queue":false,"wintergrasp":{"area":1,"controlling-faction":0,"status":0,"next":1382350068792},"tol-barad":{"area":21,"controlling-faction":0,"status":0,"next":1382349141932},"status":true,"name":"Silvermoon","slug":"silvermoon","battlegroup":"Cyclone / Wirbelsturm","locale":"en_GB","timezone":"Europe/Paris"}

At the moment I can access the main array, but don't seem to be able to access sub-arrays without copying the whole thing to another new variable which seems wasteful. I'd like to be able to do something like

import urllib2
import json

req = urllib2.Request("http://eu.battle.net/api/wow/realm/status", None, {})
opener = urllib2.build_opener()
f = opener.open(req)

x = json.load(f)  # open the file and read into a variable

# search and find the Silvermoon server
silvermoon_array = ????

# print the population
print silvermoon_array.????

# access the Wintergrasp sub-array
wintergrasp_sub = ????
print wintergrasp_sub.????  # print the controlling-faction variable

This would really help me get to grips with how to access other things.

Answer 1

Python's interactive mode is a great way to progressively explore structured data. It's easy to find how to access, say, the silvermoon server data:

>>> data=json.load(urllib2.urlopen("http://eu.battle.net/api/wow/realm/status"))
>>> type(data)
<type 'dict'>
>>> data.keys()
[u'realms']
>>> type(data['realms'])
<type 'list'>
>>> type(data['realms'][0])
<type 'dict'>
>>> data['realms'][0].keys()
[u'status', u'wintergrasp', u'battlegroup', u'name', u'tol-barad', u'locale', u'queue', u'timezone', u'type', u'slug', u'population']
>>> data['realms'][0]['name']
u'Aegwynn'
>>> [realm['name'] for realm in data['realms']].index('Silvermoon')
212
>>> silvermoon= data['realms'][212]
>>> silvermoon['population']
u'high'
>>> type(silvermoon['wintergrasp'])
<type 'dict'>
>>> silvermoon['wintergrasp'].keys()
[u'status', u'next', u'controlling-faction', u'area']
>>> silvermoon['wintergrasp']['controlling-faction']
>>> silvermoon['population']
u'high'

If you don't know about them yet, you should read up on dictionary.keys , list.index and list comprehensions to understand what's going on.

After figuring out the structure of the data, you can finally rewrite the data access to be a bit more readable and efficient:

realms= data['realms']
realm_from_name= dict( [(realm['name'], realm) for realm in realms])
print realm_from_name['Silvermoon']['population']
print realm_from_name['Silvermoon']['wintergrasp']['controlling-faction']

As to copying the array to another variable being wasteful, you should know that python passes value by reference . That means that there's no copying involved when you assign something to a new variable. Here's a simple explanation of passing by value vs passing by reference

Finally, you seem to be excessively worried with performance. Python's philosophy is get it right first, optimize later . When you have this working correctly and if you need better performance, optimize it, if it's worth the time.

Answer 2

This does what you want:

# -*- coding: utf-8 -*-
import json
import urllib2

def searchListOfDicts(listOfDicts, attr, value):
    """
    Loops through a list of dictionaries and returns matching attribute value pair
    You can also pass it slug, silvermoon or type, pve
    returns a list containing all matching dictionaries 
    """
    matches = [record for record in listOfDicts if attr in record and record[attr] == value]
    return matches

def myjsonDict():
    """
    Opens url, grabs json and puts it inside a dictionary
    """
    req = urllib2.Request("http://eu.battle.net/api/wow/realm/status", None, {})
    opener = urllib2.build_opener()
    f = opener.open(req)
    json_dict = json.load(f)
    return json_dict

jsonDict = myjsonDict()
#we want to search inside realms list
silverMoonServers = searchListOfDicts(jsonDict["realms"], "name", "Silvermoon")
#access first dictionary that matched "name, Silvermoon" query
print silverMoonServers[0]
print silverMoonServers[0]["wintergrasp"]
print silverMoonServers[0]["wintergrasp"]["controlling-faction"]

Answer 3

In Python, json.loads maps json objects to python dictionaries, and Arrays to list , so further manipulation is done just like with regular python dict and list structures.

Here is how you can do it with requests and lamdbas :

    import json
    import requests

    response = requests.get("http://eu.battle.net/api/wow/realm/status")
    json_data = json.loads(response.text)

    # loop through the list of realms to find the one you need (realm_name)
    get_realm = lambda realm_name, jd: [r for r in jd['realms'] 
                                        if r['name'] == realm_name]

    # extract data you need, if there is a match in the list of realms,
    # return None otherwise
    get_your_data = lambda realm: (
        realm[0]['name'],
        realm[0]['wintergrasp']['controlling-faction']
    ) if realm else None

    print get_your_data(get_realm('Silvermoon', json_data))