How can I convert/transform a JSON tree structure to a merkle tree

Question

I'm running a web server, where I receive data in JSON format and planning to store it in a NoSQL database. Here is an example:

data_example = {
    "key1": "val1",
    "key2": [1, 2, 3],
    "key3": {
        "subkey1": "subval1",
        .
        .
        }
      }

I had thoughts about using a Merkle tree to represent my data since JSON is also a tree-like structure.

Essentially, what I want to do is to store my data in (or as) a more secure decentralized tree-like structure. Many entities will have access to create, read, update or delete (CRUD) a record from it. These CRUD operations will ideally need to be verified from other entities in the network, which will also hold a copy of the database. Just like in blockchain.

I'm having a design/concept problem and I'm trying to understand how can I turn my JSON into a Merkle tree structure. This is my Node class:

class Node:
    """ class that represents a node in a merkle tree"""
    def __init__(data):
          self.data = data
          self.hash = self.calculate_some_hash()  # based on the data or based on its child nodes

I'm interested in the conception/design of this as I couldn't figure out how this can work. Any idea how to save/store my data_example object in a Merkle tree? (is it possible?)

Answer 1

You can create a Merkle Tree by first converting your dictionary to a class object form, and then recursively traverse the tree, hashing the sum of the child node hashes. Since a Merkle Tree requires a single root node, any input dictionaries that have more than one key at the topmost level should become the child dictionary of an empty root node (with a default key of None ):

data_example = {
  "key1": "val1",
  "key2": [1, 2, 3],
  "key3": {
     "subkey1": "subval1",
     "subkey2": "subval2",
     "subkey3": "subval3",
    }
}
class MTree:
   def __init__(self, key, value):
      self.key, self.hash = key, None
      self.children = value if not isinstance(value, (dict, list)) else self.__class__.build(value, False) 
   def compute_hashes(self):
       #build hashes up from the bottom
       if not isinstance(self.children, list):
          self.hash = hash(self.children)
       else:
          self.hash = hash(sum([i.compute_hashes() for i in self.children]))
       return self.hash
   def update_kv(self, k, v):
      #recursively update a value in the tree with an associated key
      if self.key == k:
         self.children = v
      elif isinstance(self.children, list):
         _ = [i.update_kv(k, v) for i in self.children]
   def update_tree(self, payload):
      #update key-value pairs in the tree from payload
      for a, b in payload.items():
         self.update_kv(a, b)
      self.compute_hashes() #after update is complete, recompute the hashes
   @classmethod
   def build(cls, dval, root=True):
       #convert non-hashible values to strings
       vals = [i if isinstance(i, (list, tuple)) else (None, i) for i in getattr(dval, 'items', lambda :dval)()]
       if root:
          if len(vals) > 1:
             return cls(None, dval)
          return cls(vals[0][0], vals[0][-1])
       return [cls(a, b) for a, b in vals]  
   def __repr__(self):
      return f'{self.__class__.__name__}({self.hash}, {repr(self.children)})'        

---

tree = MTree.build(data_example) #create the basic tree with the input dictionary
_ = tree.compute_hashes() #get the hashes for each node (predicated on its children)
print(tree)

Output:

MTree(-1231139208667999673, [MTree(-8069796171680625903, 'val1'), MTree(6, [MTree(1, 1), MTree(2, 2), MTree(3, 3)]), MTree(-78872064628455629, [MTree(-8491910191379857244, 'subval1'), MTree(1818926376495655970, 'subval2'), MTree(1982425731828357743, 'subval3')])]) 

---

Updating the tree with the contents from a payload:

tree.update_tree({"key1": "newVal1"})

Output:

MTree(1039734050960246293, [MTree(5730292134016089818, 'newVal1'), MTree(6, [MTree(1, 1), MTree(2, 2), MTree(3, 3)]), MTree(-78872064628455629, [MTree(-8491910191379857244, 'subval1'), MTree(1818926376495655970, 'subval2'), MTree(1982425731828357743, 'subval3')])])