Combining list of list elements from JSON

Question

Sample JSON:

{
  "data": [
    {
      "ids": [1, 2, 3],
      "category": ["a", "b"]
    },
    {
      "ids": [3, 4, 5, 6],
      "category": ["a","c","d"]
    }
  ]
}

I need to combine ids and make a unique id list.

Solution 1:

data = json.loads(str)["data"]

unique_id_set = set()

for d in data:
    for id in d["ids"]:
        unique_id_set.add(id)

unique_ids = list(unique_id_set)

Solution 2:

ids = []
for d in data:
    ids.extend(d["ids"])

unique_ids = list(set(ids))

Solution 2 reduces time complexity but is list to set conversion costly?

Is there any direct efficient JSON utility to fetch values like in the above scenario?

Answer 1

Both of your approaches are O(total number of ids in the entire data structure), so you should feel free to use whichever one you find more aesthetically pleasing.

Solution 2 reduces time complexity...

I disagree. Solution 2 may have one less explicit for loop, but that doesn't make it any more efficient, because list.extend is O(N), compared to set.add 's O(1).

...but is list to set conversion costly?

It's O(N), so it doesn't make the time complexity of your solution any worse.

Is there any direct efficient JSON utility to fetch values like in the above scenario?

Not in the stdlibs. The only json utility included in Python is json , and none of the methods in that module have anything to do with manipulating the structure of your data after it's already been loaded. Once the data is out of the file and into an object, Python doesn't consider it "a JSON" any more -- it's just a collection of dicts/lists/strings/numbers.

... And if you're thinking "In that case, is there any direct efficient dict/list/string/number utility to fetch values like these?", not that I'm aware of. It's easy enough to write your own logic in three or four lines, so there's not much demand for such a utility.

Answer 2

Look at jsonmerge ( https://pypi.org/project/jsonmerge/ ). You can specify your desired output schema and merge the input according to it.