How to parse complex json in python 2.7.5?

Question

I trying to list the names of my puppet classes from a Puppet Enterprise 3.7 puppet master, using Puppet's REST API.

Here is my script:

#!/usr/bin/env python

import requests
import json

url='https://ppt-001.example.com:4433/classifier-api/v1/groups'
headers = {"Content-Type": "application/json"}
data={}
cacert='/etc/puppetlabs/puppet/ssl/certs/ca.pem'
key='/etc/puppetlabs/puppet/ssl/private_keys/ppt-001.example.com.pem'
cert='/etc/puppetlabs/puppet/ssl/certs/ppt-001.example.com.pem'
result = requests.get(url,
        data=data, #no data needed for this request
        headers=headers, #dict {"Content-Type":"application/json"}
        cert=(cert,key), #key/cert pair 
        verify=cacert
        )
print json.dumps( result.json(), sort_keys=True, indent=4, separators=(',', ': ')) 
for i in result.json:
    print i

Here is the error message I get when I execute the script:

Traceback (most recent call last):
  File "./add-group.py", line 42, in <module>
    for i in result.json:
TypeError: 'instancemethod' object is not iterable

Here is a sample of the data I get back from the REST API:

[
    {
        "classes": {},
        "environment": "production",
        "environment_trumps": false,
        "id": "00000000-0000-4000-8000-000000000000",
        "name": "default",
        "parent": "00000000-0000-4000-8000-000000000000",
        "rule": [
            "and",
            [
                "~",
                "name",
                ".*"
            ]
        ],
        "variables": {}
    },
    {
        "classes": {
            "puppet_enterprise": {
                "certificate_authority_host": "ppt-001.example.com",
                "console_host": "ppt-001.example.com",
                "console_port": "443",
                "database_host": "ppt-001.example.com",
                "database_port": "5432",
                "database_ssl": true,
                "mcollective_middleware_hosts": [
                    "ppt-001.example.com"
                ],
                "puppet_master_host": "ppt-001.example.com",
                "puppetdb_database_name": "pe-puppetdb",
                "puppetdb_database_user": "pe-puppetdb",
                "puppetdb_host": "ppt-001.example.com",
                "puppetdb_port": "8081"
            }
        },
        "environment": "production",
        "environment_trumps": false,
        "id": "52c479fe-3278-4197-91ea-9127ba12474e",
        "name": "PE Infrastructure",
        "parent": "00000000-0000-4000-8000-000000000000",
        "variables": {}
    },
.
.
.

How should I go about access the name key and getting the values like default and PE Infrastructure ?

I have read the other answers here on SO saying that one should use json.loads() and I have tried using parsed_json = json.loads(result.json()) but results in this error message:

Traceback (most recent call last):
  File "./add-group.py", line 38, in <module>
    parsed_json = json.loads(result.json())
  File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.7/json/decoder.py", line 365, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: expected string or buffer

Answer 1

print json.dumps( result.json(), sort_keys=True, indent=4, separators=(',', ': ')) the first parameter of json.dumps must be a string or buffer, as stated by the TypeError your getting ( TypeError: expected string or buffer ).

Your variable result is an instance of Response , and the method .json() will return a dictionary. Since you're passing the result of .json() to json.dumps() , you're getting an error. You could either just use result.json() which is already a dictionary corresponding to your response, or change your json.dumps line to print json.dumps( result.text, sort_keys=True, indent=4, separators=(',', ': ')) where result.text is your JSON result as a string/unicode.

After the change, to access something like the name attribute, you could do something like:

for item in r.json():
    try:
        print item['name']
    expect KeyError:
        print "There is no 'name' attribute"