How to get inverse operation of flatten for objects using jq?

Question

Through another SO thread, I found how to "flatten" JSON using jq. I am looking how to do the inverse, ie, "unflatten" it.

Original JSON has mixed levels in the hierarchy, eg:

{
  "X1": {
    "X1o1": {
      "X1o1o1": "abc",
      "X1o1o2": "def"
    },
    "X1o2" : {
      "X1o2o1" : "ghi",
      "X1o2o2": "jkl"
    },
    "X1o3": "mno"
  },
  "X2": "pqr"
}

As https://jqplay.org/s/Eo6h_3V8PO shows, for working with other colleagues, I need to "flatten" that structure () using

reduce (tostream|select(length==2)) as $i ({}; .[[$i[0][]|tostring]|join("_")] = $i[1] )

to produce:

{
  "X1_X1o1_X1o1o1": "abc",
  "X1_X1o1_X1o1o2": "def",
  "X1_X1o2_X1o2o1": "ghi",
  "X1_X1o2_X1o2o2": "jkl",
  "X1_X1o3": "mno",
  "X2": "pqr"
}

What I need help is how to transform such an output JSON back to the original form using jq?

I see that the split only works with strings as in tostring | split("_") tostring | split("_") .

Any pointers and/or examples to guide me in the right direction would be much appreciated.

Answer 1

With your sample input, the filter:

reduce to_entries[] as $kv ({}; setpath($kv.key|split("_"); $kv.value))

produces:

{
  "X1": {
    "X1o1": {
      "X1o1o1": "abc",
      "X1o1o2": "def"
    },
    "X1o2": {
      "X1o2o1": "ghi",
      "X1o2o2": "jkl"
    },
    "X1o3": "mno"
  },
  "X2": "pqr"
}

Answer 2

There are two tools you may want to know about. The first is a program called gron which can flatten JSON into valid JavaScript (which can be unflattened back into the original JSON). It would transform your original into this:

$ gron /tmp/foo.json
json = {};
json.X1 = {};
json.X1.X1o1 = {};
json.X1.X1o1.X1o1o1 = "abc";
json.X1.X1o1.X1o1o2 = "def";
json.X1.X1o2 = {};
json.X1.X1o2.X1o2o1 = "ghi";
json.X1.X1o2.X1o2o2 = "jkl";
json.X1.X1o3 = "mno";
json.X2 = "pqr";

and unflatten that back into JSON:

$ gron /tmp/foo.json | gron -u
{
  "X1": {
    "X1o1": {
      "X1o1o1": "abc",
      "X1o1o2": "def"
    },
    "X1o2": {
      "X1o2o1": "ghi",
      "X1o2o2": "jkl"
    },
    "X1o3": "mno"
  },
  "X2": "pqr"
}

The second is a Bash script called jqg that achieves similar results using jq ; its intermediate (flattened) format is valid JSON:

$ jqg . /tmp/foo.json
{
  "X1.X1o1.X1o1o1": "abc",
  "X1.X1o1.X1o1o2": "def",
  "X1.X1o2.X1o2o1": "ghi",
  "X1.X1o2.X1o2o2": "jkl",
  "X1.X1o3": "mno",
  "X2": "pqr"
}

which unflattens back into the original:

$ jqg . /tmp/foo.json | jqg -u
{
  "X1": {
    "X1o1": {
      "X1o1o1": "abc",
      "X1o1o2": "def"
    },
    "X1o2": {
      "X1o2o1": "ghi",
      "X1o2o2": "jkl"
    },
    "X1o3": "mno"
  },
  "X2": "pqr"
}

Note: I am the author of jqg .