Bash script to extract all specific key values from a unstructured JSON file

Question

I was trying to extract all the values from a specific key in the below JSON file.

{
  "tags": [
    {
      "name": "xxx1",
      "image_id": "yyy1"
    },
    {
      "name": "xxx2",
      "image_id": "yyy2"
    }
  ]
}

I used the below code to get the image_id key values.

echo new.json | jq '.tags[] | .["image_id"]'

I'm getting the below error message.

parse error: Invalid literal at line 2, column 0

I think either the JSON file is not in the proper format OR the echo command to call the Json file is wrong.

Given the above input, my intended/desired output is:

yyy1
yyy2

What needs to be fixed to make this happen?

Answer 1

When you run:

echo new.json | jq '.tags[] | .["image_id"]'

...the string new.json -- not the contents of the file named new.json -- is fed to jq 's stdin, and is thus what it tries to parse as JSON text.

Instead, run:

jq -r '.tags[] | .["image_id"]' <new.json

...to directly open new.json connected to the stdin of jq (and, with -r , to avoid adding unwanted quotes to the output stream).

Answer 2

also, you may wanna try an alternative approach to your ask - using a walk-path unix tool for JSON: jtc . With that one your ask would look like this:

bash $ <new.json jtc -w'[tags][:][image_id]'
"yyy1"
"yyy2"
bash $ 

However, your new.json is not unstructured, oppositely it's well-structured. if your new.json was indeed irregular (unstructured), then the following query would work better:

bash $ <new.json jtc -w'<image_id>l:'
"yyy1"
"yyy2"
bash $ 

Answer 3

1. Your filter .tags[] | .["image_id"] .tags[] | .["image_id"]

is valid, but can be abbreviated to:

.tags[] | .image_id

or even:

.tags[].image_id

2. If you want the values associated with the "image_id" key, wherever that key occurs, you could go with:

   .. | objects | select(has("image_id")) | .image_id

Or, if you don't mind throwing away false and null values:

.. | .image_id? // empty