merge aws cli results

Question

I am trying to figure out how to join results of aws cli paginated output. Say the query looks like so: aws ec2 describe-snapshots --output json --max-items 100 --starting-token ABC-123 Using the next token I am generating N files: out.0.json, out.1.json, ...

Each such file looks like so:

{
"Snapshots": [
    {
        "Description": "hvm/ubuntu-vivid-amd64-server-20160119",
        "Encrypted": false,
        "OwnerId": "0123456789",
        "Progress": "100%",
        "SnapshotId": "snap-0001",
        "StartTime": "2016-01-19T19:24:47+00:00",
        "State": "completed",
        "VolumeId": "vol-0001",
        "VolumeSize": 8
    },
    {
        "Description": "ebs/ubuntu-trusty-i386-server-20151007",
        "Encrypted": false,
        "OwnerId": "0123456789",
        "Progress": "100%",
        "SnapshotId": "snap-1234",
        "StartTime": "2015-10-07T23:09:20+00:00",
        "State": "completed",
        "VolumeId": "vol-1235",
        "VolumeSize": 8
    },
    {
        "Description": "hvm-io1/ubuntu-precise-amd64-server-20150512",
        "Encrypted": false,
        "OwnerId": "0123456789",
        "Progress": "100%",
        "SnapshotId": "snap-1235",
        "StartTime": "2015-05-13T02:17:25+00:00",
        "State": "completed",
        "VolumeId": "vol-1235",
        "VolumeSize": 8
    },
    .......

How do I merge the list of snapshots from all files into one? I tried jq with merge, but nothing seems to work

Answer 1

Here's a solution that's both simple and efficient. It assumes that out.*.json identifies the relevant files in the required order if any.

jq -n '{Snapshots: [inputs.Snapshots] | add}' out.*.json

Note that the -n option is necessary here.