jq group by and Increment at same time

Question

Sample input1.json

[{
    "organizationId": "org1",
    "status": "UP",
    "server": "server1",
    "somekey1" : "somevalue1"
}, 
{
    "organizationId": "org1",
    "status": "DOWN",
    "server": "server2",
    "somekey2" : "somevalue2"
},
{
    "organizationId": "org1",
    "status": "DOWN",
    "server": "server3"
},
{
    "organizationId": "org2",
    "server": "server1",
    "status": "UP",
    "somekey2" : "somevalue2"
},
{
    "organizationId": "org2",
    "server": "server2",
    "status": "UP",
    "somekey4" : "somevalue4"
}]

Expected output.json

[{
        "organizationId": "org1",
        "server1": "UP",
        "server2": "DOWN",
        "server3": "DOWN",
        "Down_Count": 2,
        "somekey2" : "somevalue2"
    },
    {
        "organizationId": "org2",
        "server1": "UP",
        "server2": "UP",
        "Down_Count": 0,
       "somekey2" : "somevalue2"
    }]

The input is an array object of two files. My objective is

1. to increment and populate the new field Down_Count by grouping by organizationId . I can have multiple records with same organizationId . Hence, the count should be incremented that number of times subject to status DOWN .
2. Merge server and status fields. For example "server":"server1", "status":"down" should come as server1:down directly in final output. I can try this one independently using jq '.[] |= . + {(.server):.status}' jq '.[] |= . + {(.server):.status}'
3. all the keys are not available in all the objects. Hence, I should be able to add the keys I want to the final output. Lets assume I know the key name I want to add. In the final output, I have added only "somekey2" : "somevalue2"

I'm finding some difficulties to achieve all these together using jq cmd line processor. Any suggestions please?

Answer 1

jq solution:

jq 'group_by(.organizationId) 
    | map(reduce .[] as $o ({"Down_Count" : 0};
              if $o["status"] == "DOWN" then .Down_Count += 1 else . end
              | . + { ($o["server"]) : $o["status"],
                          "organizationId" : $o["organizationId"] }
          ))' input.json

The output:

[
  {
    "Down_Count": 2,
    "server1": "UP",
    "organizationId": "org1",
    "server2": "DOWN",
    "server3": "DOWN"
  },
  {
    "Down_Count": 0,
    "server1": "UP",
    "organizationId": "org2",
    "server2": "UP"
  }
]

Answer 2

Here's another approach you could take:

group_by(.organizationId) | map(
    reduce ([., [range(length)]] | transpose[]) as [$o,$i] (
        {
            organizationId: .[0].organizationId,
            Down_Count: (map(select(.status=="DOWN")) | length)
        };
        reduce $keys[] as $k (
            .["server\($i+1)"] = $o.status;
            if $o | has($k) then .[$k] = $o[$k] else . end
        )
    )
)

Just pass in an array of keys named $keys you want to be copied over. If it exists in one of the objects, it will be copied to the result object.

$ cat input.json
[{
    "organizationId": "org1",
    "status": "UP",
    "server": "server1",
    "somekey1" : "somevalue1"
}, 
{
    "organizationId": "org1",
    "status": "DOWN",
    "server": "server2",
    "somekey2" : "somevalue2"
},
{
    "organizationId": "org1",
    "status": "DOWN",
    "server": "server3"
},
{
    "organizationId": "org2",
    "server": "server1",
    "status": "UP",
    "somekey2" : "somevalue2"
},
{
    "organizationId": "org2",
    "server": "server2",
    "status": "UP",
    "somekey4" : "somevalue4"
}]

$ cat program.jq
group_by(.organizationId) | map(
    reduce ([., [range(length)]] | transpose[]) as [$o,$i] (
        {
            organizationId: .[0].organizationId,
            Down_Count: (map(select(.status=="DOWN")) | length)
        };
        reduce $keys[] as $k (
            .["server\($i+1)"] = $o.status;
            if $o | has($k) then .[$k] = $o[$k] else . end
        )
    )
)

$ jq --argjson keys '["somekey2"]' -f program.jq input.json
[
  {
    "organizationId": "org1",
    "Down_Count": 2,
    "server1": "UP",
    "server2": "DOWN",
    "somekey2": "somevalue2",
    "server3": "DOWN"
  },
  {
    "organizationId": "org2",
    "Down_Count": 0,
    "server1": "UP",
    "somekey2": "somevalue2",
    "server2": "UP"
  }
]