Python, JSON and Multiple Objects

Question

Python3 coding:

I am trying to extract fields from a web JSON response but just can't see how to iterate the objects. The JSON has some standard entries, then fires into a list of objects and back out to more standard entries. I have summarised this below:

{
  "accepted_count": 38526,
  "cpu_total": 6,
  "cpus": [
    {
      "device_id": 0,
      "hashrate": 61382378,
    },
    {
      "device_id": 1,
      "hashrate": 61439706,
    },
    {
      "device_id": 2,
      "hashrate": 61433822,
    },
    {
      "device_id": 3,
      "hashrate": 61229485,
    },
    {
      "device_id": 4,
      "hashrate": 61183044,
    },
    {
      "device_id": 5,
      "hashrate": 23950340,
    }
  ],
  "hashrate": 330618775,
  "invalid_count": 0,
  "os": "win",
  }
}

I can load this data from the web site using (webURL is the web site returning the json):

s = requests.Session()
jsonData = s.get(webURL)
data = jsonData.json()

The problem is if I ask for data['hashrate'] I just get the hashrate value at the very bottom (330618775). I need to work out how to extract each of the hashrate values in the objects and be able to relate it to the device_id value in the same object.

I have tried the likes of:

data[0]['hashrate']

but I just get an error: KeyError: '0'

Same if the key is 1, 2 etc

I am at a loss, any help would be appreciated

Thanks in advance Guy

Answer 1

but I just get an error: KeyError: '0' data[0]['hashrate']

The 'hashrate' you're trying to find are in a array on the cpus key, so you'll need to loop over the array:

---

I've created a file called input.json and a simple script like so:

import json

with open('input.json') as f:
    d = json.load(f)

    for cpu in d['cpus']:
        print("ID: {:d} || HashRate: {:d}".format(cpu['device_id'], cpu['hashrate']))

This will produce:

ID: 0 || HashRate: 61382378
ID: 1 || HashRate: 61439706
ID: 2 || HashRate: 61433822
ID: 3 || HashRate: 61229485
ID: 4 || HashRate: 61183044
ID: 5 || HashRate: 23950340

You can easily sum up the total hashrate if needed

Answer 2

When searching for techniques to access a json data structure in python, it may be more helpful to instead learn about python collections such as list s and dict s.

When parsed, in python, they are nothing more than that.

So if you look at your top level collection, it is represented by curly braces or {} . This is the representation of a python dict , basically a {key: value} list. To access elements, you reference the key who's value you want to access.

print(data["cpus"])

In your example the keys are "accepted_count" , "cpu_total" , "cpus" , "hashrate" , "invalid_count" , and "os" respectively.

The value of the "cpus" key is represented by [] which represents a python list . A list is accessed by its integer index.

cpus = data["cpus"]
print(cpus[0])

Finally, each element in the list is a dict consisting of 2 keys, which as you might have guessed, are accessed by referencing the key you want.

cpus = data["cpus"]
first_cpu = cpus[0]
print(first_cpu["hashrate"])

Combined:

data = {
    "accepted_count": 38526,
    "cpu_total": 6,
    "cpus": [
        {
            "device_id": 0,
            "hashrate": 61382378,
        },
        {
            "device_id": 1,
            "hashrate": 61439706,
        },
        {
            "device_id": 2,
            "hashrate": 61433822,
        },
        {
            "device_id": 3,
            "hashrate": 61229485,
        },
        {
            "device_id": 4,
            "hashrate": 61183044,
        },
        {
            "device_id": 5,
            "hashrate": 23950340,
        }
    ],
    "hashrate": 330618775,
    "invalid_count": 0,
    "os": "win"
}

print(data['cpus'][0]['hashrate'])