How to handle missing keys in list of JSON objects in Python

Question

I have a list of dicts that are JSON objects, like this:

[{'addressId': 12345, 'city': 'London', 'country': 'UK'}, {'addressId': 67890, 'city': 'Berlin'}

For some of the entries there are some JSON keys missing (not just the values) - in the example above the second address is missing the country. This causes problems down the line when I want to write this data to MySQL.

So I think what I want to do is add the missing keys with empty values. I'm wondering (a) if this is the right approach, and (b) how I would go about doing that.

FWIW, we're talking about a few million rows.

Any hints are greatly appreciated.

Answer 1

Whether or not this is the correct approach depends on the use case down the line. You could simply loop through each element in the list, and, if the key doesn't exist, add it with a default value:

for obj in arr:
    if key1 not in obj:
       obj[key1] = # default value
    # and continue for other keys

Answer 2

I would recommend just doing dictionary.get() for each key at the point of putting the info into the database, that way you can add a default value there if it's missing.

objects = [{'addressId': 12345, 'city': 'London', 'country': 'UK'}, {'addressId': 67890, 'city': 'Berlin'}]

for object in objects:
   country = object.get('country', None)
   city = object.get('city', None)

The default return value for .get() if it doesn't find the key, is None. But I included it in there to show you can put anything there.

Answer 3

If data is not uniform or difficult to present in tabular format than mongodb is the better option. if strict to MYSQL than As Tob suggested ,make the default values .