使用 jq 展平 JSON 文档

Question

I'm considering the following array of JSON objects:

[
  {
    "index": "index1",
    "type": "type1",
    "id": "id1",
    "fields": {
      "deviceOs": [
        "Android"
      ],
      "deviceID": [
        "deviceID1"
      ],
      "type": [
        "type"
      ],
      "country": [
        "DE"
      ]
    }
  },
  {
    "index": "index2",
    "type": "type2",
    "id": "id2",
    "fields": {
      "deviceOs": [
        "Android"
      ],
      "deviceID": [
        "deviceID2"
      ],
      "type": [
        "type"
      ],
      "country": [
        "US"
      ]
    }
  }
]

and I would like to flatten it to get:

[
  {
    "index": "index1",
    "type": "type",
    "id": "id1",
    "deviceOs": "Android",
    "deviceID": "deviceID1",
    "country": "DE"
  },
  {
    "index": "index2",
    "type": "type",
    "id": "id2",
    "deviceOs": "Android",
    "deviceID": "deviceID2",
    "country": "US"
  }
]

I'm trying to work with jq but I fail to flatten the "fields" . How should I do it? At the moment I'm interested in command-line tools, but I'm open to other suggestions as well.

Answer 1

This one was a tricky one to craft.

map
(
    with_entries(select(.key != "fields"))
    +
    (.fields | with_entries(.value = .value[0]))
)

Let's break it down and explain the bits of it

1. For every item in the array...

    map(...) 

2. Create a new object containing the values for all except the fields property.

    with_entries(select(.key != "fields")) 

3. Combine that with...

    + 

4. Each of the fields projecting each of the values to the first item of each array

    (.fields | with_entries(.value = .value[0])) 

Answer 2

You can use this filter:

[.[] | {index: .index, type: .type, id: .id, deviceOs: .fields.deviceOs[],deviceID: .fields.deviceID[],country: .fields.country[]}]

You can test here https://jqplay.org

Answer 3

Here are some variations that start by merging .fields into the containing object with + and then flattening the array elements. First we take care of .fields with

  .[]
| . + .fields
| del(.fields)

that leaves us with objects which look like

{
  "index": "index1",
  "type": [
    "type"
  ],
  "id": "id1",
  "deviceOs": [
    "Android"
  ],
  "deviceID": [
    "deviceID1"
  ],
  "country": [
    "DE"
  ]
}

then we can flatten the keys multiple ways. One way is to use with_entries

| with_entries( .value = if .value|type == "array" then .value[0] else .value end )

another way is to use reduce and setpath

| . as $v
| reduce keys[] as $k (
    {};
    setpath([$k]; if $v[$k]|type != "array" then $v[$k] else $v[$k][0] end)
  )

Answer 4

There's a tool called gron and you can pipe that json in, to get something like this.

$ gron document.json
json = [];
json[0] = {};
json[0].fields = {};
json[0].fields.country = [];
json[0].fields.country[0] = "DE";
json[0].fields.deviceID = [];
json[0].fields.deviceID[0] = "deviceID1";
json[0].fields.deviceOs = [];
json[0].fields.deviceOs[0] = "Android";
json[0].fields.type = [];
json[0].fields.type[0] = "type";
json[0].id = "id1";
json[0].index = "index1";
json[0].type = "type1";
json[1] = {};
json[1].fields = {};
json[1].fields.country = [];
json[1].fields.country[0] = "US";
json[1].fields.deviceID = [];
json[1].fields.deviceID[0] = "deviceID2";
json[1].fields.deviceOs = [];
json[1].fields.deviceOs[0] = "Android";
json[1].fields.type = [];
json[1].fields.type[0] = "type";
json[1].id = "id2";
json[1].index = "index2";
json[1].type = "type2";

Answer 5

Similar to the @jq170727 anwser:

jq 'map(. + (.fields | with_entries(.value |= .[])) | del(.fields))'

(assuming no field inside .fields is itself called .fields ).

The |with_entries(.value|=.[]) part is to flatten the value arrays in .fields -- beware only the first item is preserved. .value|=join(", ") could be used to join multiple string values into one.