如何使用`jq`通过嵌套值过滤这个json并打印父键标识符？

Question

Suppose I have this json

{
  "sha256:0085b5379bf1baeb4a430128782440fe636938aa739f6a5ecc4152a22f19b08b": {
    "imageSizeBytes": "596515805",
    "layerId": "",
    "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
    "tag": [
      "python-3-toolchain-0.1.2"
    ],
    "timeCreatedMs": "1564631021992",
    "timeUploadedMs": "1564631067325"
  },
  "sha256:1ec7631f74a3d6d37bf9194c13854f33315260ae1f27347263dd0a8974ee82bb": {
    "imageSizeBytes": "513574770",
    "layerId": "",
    "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
    "tag": [
      "python-2-toolchain-latest"
    ],
    "timeCreatedMs": "1535447023647",
    "timeUploadedMs": "1535447042373"
  }
}

I want to select the image information (as well as the sha256 digest) with certain tag. Example: I want to select only tag == "python-2-toolchain-latest" , so it prints this json (with json reformat)

 {
    "digest": "sha256:1ec7631f74a3d6d37bf9194c13854f33315260ae1f27347263dd0a8974ee82bb",
    "imageSizeBytes": "513574770",
    "layerId": "",
    "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
    "tag": [
      "python-2-toolchain-latest"
    ],
    "timeCreatedMs": "1535447023647",
    "timeUploadedMs": "1535447042373"
  }

I have tried various approach, and being stuck at how to reference the sha256 key information.

Answer 1

A possible jq program to accomplish your goal:

# Embed the final result into an array to get a valid JSON output
[
    # Convert the input object into a list of { key, value } objects
    to_entries[]

    # Keep only the objects that contain the desired tag
    # The .tag field may contain multiple tags and the desired one can be at any position
    | select(.value.tag | contains(["python-2-toolchain-latest"]))

    # Add the key into the value object into the .digest property
    | .value.digest = .key

    # Keep only the values (the modified objects)
    | .value

# That's all, folks
]

Try it online!

Answer 2

Here's a straightforward and concise but efficient solution:

keys_unsorted[] as $k
| .[$k] as $value
| select($value.tag[0] ==  "python-2-toolchain-latest")
| {digest: $k} + $value

Answer 3

here is what i worked up. i assumed that the tag array could contain more than a single entry...

.
|to_entries[]
|.key as $k
|.value as $v
|.value.tag[]
|select(.=="python-2-toolchain-latest")
[ { "digest": ($k) }, $v ] | add

after seeing the peak answer, I like the last line like this better:

[ { "digest": ($k) } + $v ]

If the tag can occur twice, then this would output the same record twice. there must be a better way to simply check if "python-2-toolchain-latest" is in the tag[] array. My jq foo is not strong enough.

Answer 4

for those, who are open to alternatives, here's how the same JSON manipulation is achievable with a walk-path based unix utility jtc :

bash $ <file.json jtc -w'[:]<D>k<tag>l<python-2-toolchain-latest>[-2]' -T'{"digest":{D}",{}}'
{
   "digest": "sha256:1ec7631f74a3d6d37bf9194c13854f33315260ae1f27347263dd0a8974ee82bb",
   "imageSizeBytes": "513574770",
   "layerId": "",
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "tag": [
      "python-2-toolchain-latest"
   ],
   "timeCreatedMs": "1535447023647",
   "timeUploadedMs": "1535447042373"
}

Note: lexeme <tag>l is not required if searchable context in the next lexeme uniquely identifies the tag. if so, then the lexeme could be omitted

PS> Disclosure: I'm the creator of the jtc - shell cli tool for JSON operations

Answer 5

I would use

.
| to_entries
| .[]
| select(.value.tag | contains(["python-2-toolchain-latest"]))
| { digest: .key } + .value

• I put your example data in a file I named pratama-1.json .
• I ran the following command and got this output.

   $ jq '. | to_entries | .[] | select(.value.tag | contains(["python-2-toolchain-latest"])) | { digest: .key } + .value' pratama-1.json { "digest": "sha256:1ec7631f74a3d6d37bf9194c13854f33315260ae1f27347263dd0a8974ee82bb", "imageSizeBytes": "513574770", "layerId": "", "mediaType": "application/vnd.docker.distribution.manifest.v2+json", "tag": [ "python-2-toolchain-latest" ], "timeCreatedMs": "1535447023647", "timeUploadedMs": "1535447042373" }

Breaking this down

You can think of a jq program as analogous to a sh pipeline, except: - with sh (or bash ), each stage of the pipeline (that is, each command between | s) has a stream of bytes (often just text) as its input and output; - with jq , each stage has a stream of JSON values as its inputs and outputs.

1. The simplest jq program is simply . :
   • If the input is the JSON value true , the output is true .
   • If the input is the JSON value [ true, 3.1416, "foo" ] (an array value), the output is the same.
   • If the inputs are the three JSON values true , 3.1416 , and "foo" (a boolean, a number, and a string), then the three outputs will be that boolean, that number, and that string.
2. At the beginning of my script, the . just represents the input value, which in this case is that JSON object you included in your question.
3. The next stage is to_entries :
   • It converts a JSON object into a JSON array of JSON objects. For an input like:

      { "a": 3.1416, "b": false }

     into an array like this:

      [ { "key": "a", "value": 3.1416 }, { "key": "b", "value": false } ]

4. The next stage is .[] , which is a jq operator that turns one JSON value into many:
   • If the input is a single JSON array like [ true, 3.1416, "foo" ] , the three ouptuts are the JSON values true , 3.1416 , and "foo" .
   • In our case, it unwraps the JSON array around all those key-value objects so we can many output values instead of one JSON array output value.
5. The next stage is the select(…) :
   • For each input value, it evaluates the expression in its parentheses and "passes along" that input as an output if the expression is true.
   • For example, for the three inputs true , 3.1416 , and "foo" , select(type == "string") would have only one ouput: the string "foo" .
   • My select(…) has two inputs:
     1. A JSON object { "key": "sha256:008…", "value": { "imageSizeBytes": … } } AND
     2. A JSON object { "key" :"sha256:1ec…", "value": { "imageSizeBytes": … } } .
   • Inside my select(…) , I use a subexpression that is itself a jq pipeline: .value.tag | contains(["python-2-toolchain-latest"]) .value.tag | contains(["python-2-toolchain-latest"]) :
     1. The first part .value.tag yields the value of the field with key "tag" in each of those objects. For your example data, each value is a JSON array.
     2. The contains([…]) part evalues to true for input JSON array values if all the values in its argument JSON array […] are members of that input JSON array value.
        • Some examples

        $ jq '. | contains([ "foo" ])' <<< '[ true, 3.1416, "foo" ]' true $ jq '. | contains([ "foo", true ])' <<< '[ true, 3.1416, "foo" ]' true $ jq '. | contains([ "foo", true, null ])' <<< '[ true, 3.1416, "foo" ]' false $ jq '. | contains([ "foo", true, false ])' <<< '[ true, 3.1416, "foo" ]' false $ jq '. | contains([ "foo", true, null ])' <<< '[ true, 3.1416, "foo" ]' false

   • So, the expression in my select(…) evaluates to true for each imput JSON object that has a key named "value" with a JSON sub-object value that has a key named "tag" with a JSON array value that contains an element with the JSON string value "python-2-toolchain-latest".
   • For each input where the expression inside select(…) is true , that input value becomes one of the output values.
   • For your example data, this is only the second sub-object: { "key": "sha256:1ec…", "value": { … } } .
6. The last stage of my pipeline is a jq expression that looks like a JSON object.
   • { digest: .key } says for each input value, output a JSON object value with:
     • a key named "digest" AND
     • the value of that key should be the value associated with the input JSON object's key named "key" .
     • Since our input is the JSON object { "key": "sha256:1ec…", … } , this would give us the output JSON object { "digest": "sha256:1ec…" } .
   • But we want other stuff in our output JSON object too: we want to add in the JSON object value associated with the key "value" in our input object. We get that by adding + .value .
   • The jq + operator when used with JSON object values "merges" its operands JSON objects into a single output JSON object, eg { "a": true } + { "b": 3.1416 } yields { "a": true, "b": 3.1416 } .