Is there something wrong with this json reference?

Question

I'm trying to take some JSON containing references and resolve them. I'm using the jsonref library to do it. I have reduced my problem to these two cases:

import jsonref

print(jsonref.JsonRef.replace_refs(jsonref.loads('''
{
  "foo": {
    "$ref": "#/def/bar"
  },
  "def": {
      "bar": "baz"
  }
}
''')))
# works: {'foo': 'baz', 'def': {'bar': 'baz'}}

print(jsonref.JsonRef.replace_refs(jsonref.loads('''
{
  "foo": {
    "$ref": "#/def/obj"
  },
  "def": {
    "obj": {
      "bar": "baz"
    }
  }
}
''')))
# expected: {'foo': { 'bar': 'baz'}, 'def': {'bar': 'baz'}}
# actual: AttributeError: 'generator' object has no attribute 'get'

The first one works, but the second one throws an error. Why?

Answer 1

You mean something like this?

>>> import jsonref
>>> s = '''
... {
...   "foo": {
...     "$ref": "#/def/obj"
...   },
...   "def": {
...     "obj": {
...       "bar": "baz"
...     }
...   }
... }
... '''
>>> j = jsonref.loads(s)
>>> j
{u'foo': {u'bar': u'baz'}, u'def': {u'obj': {u'bar': u'baz'}}}
>>> 

NB: never used jsonref , didn't even read the doc (!!!) so I can't tell why you get this error, but there's certainly something about the correct way to use it in the doc. But obviously (from a 30s test), jsonref.loads() already applies references replacements and jsonref.JsonRef.replace_refs() is only meant to be used on already unserialized objects, ie:

>>> s = '''
... {
...   "foo": {
...     "$ref": "#/def/obj"
...   },
...   "def": {
...     "obj": {
...       "bar": "baz"
...     }
...   }
... }
... '''
>>> import json
>>> decoded = json.loads(s) # so we get a plain python dict
>>> print(decoded)
{u'foo': {u'$ref': u'#/def/obj'}, u'def': {u'obj': {u'bar': u'baz'}}}
>>> final = jsonref.JsonRef.replace_refs(decoded)
>>> print(final)
{u'foo': {u'bar': u'baz'}, u'def': {u'obj': {u'bar': u'baz'}}}

Answer 2

@bruno's answer works. But to answer 'Why'

Lets say

a=jsonref.JsonRef.replace_refs(jsonref.loads('''
{
  "foo": {
    "$ref": "#/def/obj"
  },
  "def": {
    "obj": {
      "bar": "baz"
    }
  }
}
'''))

It is because the object is not yet a dict.

type(a['foo'])# returns JsonRef

Also this will break

import json
json.dumps(a) # Gives error

So one work around could be,

jsonref.JsonRef.replace_refs(json.loads(json.dumps(jsonref.loads('''
{
  "foo": {
    "$ref": "#/def/obj"
  },
  "def": {
    "obj": {
      "bar": "baz"
    }
  }
}
'''), default=dict)))

This coneverts any JsonRef object to dict . Of course any other object will also be Json serialized. So care must be taken.

Answer 3

The accepted solves the initial confusion. In my case this answer gave a relevant clue for a subsequent problem, which is re-serializing the object back into referenceless JSON.

I settled on this:

import jsonref
import json

def ref_caster(o):
    if isinstance(o, jsonref.JsonRef):
        if isinstance(o, type(None)):
            return None
        else:
            for json_type in [ dict, str, list, float, int, bool ]:
                if isinstance(o, json_type):
                    return json_type(o)

with_ref_objs = jsonref.loads('''
{
  "foo": {
    "$ref": "#/def/obj"
  },
  "def": {
    "obj": {
      "bar": "baz"
    }
  }
}
''')
no_ref_str = json.dumps(with_ref_objs, default=ref_caster, indent=2)
print(no_ref_str)

Output:

{
  "foo": {
    "bar": "baz"
  },
  "def": {
    "obj": {
      "bar": "baz"
    }
  }
}

Answer 4

If you'd like to get the dictionary of the dereferenced schema, deref_schema_dict :

import ast
import json
import jsonref

SchemaService(object):
    def __init__(self):
        # it'd be good to use one instance of jsonloader to utilise its caching
        self.json_loader = jsonref.JsonLoader()

    def dereference_schema(self, json_schema) -> dict:
        json_ref_obj = jsonref.loads(json.dumps(json_schema), loader=json_loader)
        deref_schema_str = str(json_ref_obj)
        deref_schema_dict = ast.literal_eval(deref_schema_str)

        return deref_schema_dict