how to extract and modify inner array objects with parent object data in jq

Question

We are tying to format a json similar to this:

[
{"id": 1,
    "type": "A",
    "changes": [ 
        {"id": 12},
        {"id": 13}
    ],
    "wanted_key": "good",
    "unwanted_key": "aaa"
},
{"id": 2,
    "type": "A",
    "unwanted_key": "aaa"
},
{"id": 3,
    "type": "B",
    "changes": [
        {"id": 31},
        {"id": 32}
    ],
    "unwanted_key": "aaa",
    "unwanted_key2": "aaa"
},
{"id": 4,
    "type": "B",
    "unwanted_key3": "aaa"
},
null,
null,
{"id": 7}
]

into something like this:

[
  {
    "id": 1,
    "type": "A",
    "wanted_key": true  # every record must have this key/value
  },
  {
    "id": 12,  # note: this was in the "changes" property of record id 1
    "type": "A",  # type should be the same type than record id 1
    "wanted_key": true
  },
  {
    "id": 13,  # note: this was in the "changes" property of record id 1
    "type": "A",  # type should be the same type than record id 1
    "wanted_key": true
  },
  {
    "id": 2,
    "type": "A",
    "wanted_key": true
  },
  {
    "id": 3,
    "type": "B",
    "wanted_key": true
  },
  {
    "id": 31,  # note: this was in the "changes" property of record id 3
    "type": "B",  # type should be the same type than record id 3
    "wanted_key": true
  },
  {
    "id": 32,  # note: this was in the "changes" property of record id 3
    "type": "B",  # type should be the same type than record id 3
    "wanted_key": true
  },
  {
    "id": 4,
    "type": "B",
    "wanted_key": true
  },
  {
    "id": 7,
    "type": "UNKN",  # records without a type should have this type
    "wanted_key": true
  }
]

So far, I've been able to:

• remove null records
• obtain the keys we need with their default
• give records without a type a default type

What we are missing:

• from records having a changes key, create new records with the type of their parent record
• join all records in a single array

Unfortunately we are not entirely sure how to proceed... Any help would be appreciated.

So far our jq goes like this: del(..|nulls) | map({id, type: (.type // "UNKN"), wanted_key: (true)}) | del(..|nulls) del(..|nulls) | map({id, type: (.type // "UNKN"), wanted_key: (true)}) | del(..|nulls)

Here's our test code:

https://jqplay.org/s/eLAWwP1ha8P

Answer 1

The following should work:

map(select(values))
| map(., .type as $type | (.changes[]? + {$type}))
| map({id, type: (.type // "UNKN"), wanted_key: true})

1. Only select non-null values
2. Return the original items followed by their inner changes array (+ outer type)
3. Extract 3 properties for output

Multiple map calls can usually be combined, so this becomes:

map(
    select(values)
    | ., (.type as $type | (.changes[]? + {$type}))
    | {id, type: (.type // "UNKN"), wanted_key: true}
)

---

Another option without variables:

map(
    select(values)
    | ., .changes[]? + {type}
    | {id, type: (.type // "UNKN"), wanted_key: true}
)
# or:
map(select(values))
| map(., .changes[]? + {type})
| map({id, type: (.type // "UNKN"), wanted_key: true})

or even with a separate normalization step for the unknown type:

map(select(values))
| map(.type //= "UNKN")
| map(., .changes[]? + {type})
| map({id, type, wanted_key: true})
# condensed to a single line:
map(select(values) | .type //= "UNKN" | ., .changes[]? + {type} | {id, type, wanted_key: true})

Explanation:

1. Select only non-null values from the array
2. If type is not set, create the property with value "UNKN"
3. Produce the original array items, followed by their nested changes elements extended with the parent type
4. Reshape objects to only contain properties id, type, and wanted_key.

Answer 2

Here's one way:

map(
  select(values)
  | (.type // "UNKN") as $type
  | ., .changes[]?
  | {id, $type, wanted_key: true}
)

[
  {
    "id": 1,
    "type": "A",
    "wanted_key": true
  },
  {
    "id": 12,
    "type": "A",
    "wanted_key": true
  },
  {
    "id": 13,
    "type": "A",
    "wanted_key": true
  },
  {
    "id": 2,
    "type": "A",
    "wanted_key": true
  },
  {
    "id": 3,
    "type": "B",
    "wanted_key": true
  },
  {
    "id": 31,
    "type": "B",
    "wanted_key": true
  },
  {
    "id": 32,
    "type": "B",
    "wanted_key": true
  },
  {
    "id": 4,
    "type": "B",
    "wanted_key": true
  },
  {
    "id": 7,
    "type": "UNKN",
    "wanted_key": true
  }
]

Demo

Answer 3

Something like below should work

map( 
  select(type == "object") | 
  ( {id}, {id : ( .changes[]? .id )} ) + 
  { type: (.type // "UNKN"), wanted_key: true }
)

jq play - demo