Accessing JQ Variables Using Arguments

Question

I am trying to get values base on key of this JSON:

{
  "streams": {
    "vs-first": {
      "version": "2.33.0",
      "branch": "ewewew",
      "hash": "ewewewewe",
      "widgets": []
    },
    "vs-second": {
      "version": "1.58.0",
      "branch": "ewewew",
      "hash": "ewewew",
      "widgets": []
    },
    "vs-third": {
      "version": "1.42.0",
      "branch": "ewewew",
      "hash": "ewewe",
      "widgets": []
    },
    "vs-fourth": {
      "version": "1.58.0",
      "branch": "eewfwfef",
      "hash": "vvfffsfsf",
      "widgets": []
    },
    "vs-fifth": {
      "version": "1.39.0",
      "branch": "fvrvvsdvds",
      "hash": "vvsvdsvds",
      "widgets": [
        "1",
        "2",
        "3",
        "4"
      ]
    }
  }
}

This is my Script implementation:

jq -r '.streams|keys[]' $vsconfig | while read key ; do
    if [ $key == "[" ] || [ $key == "]" ]; then
      continue
    fi

    if [ $key == "vs-first" ]; then
      version=$(jq -r '.streams.vs-first.version' $vsconfig)
      branch=$(jq -r '.streams.vs-first.branch' $vsconfig)
      hash=$(jq -r '.streams.vs-first.hash' $vsconfig)

      filename="one_file-$version-$branch-$hash.zip"
      createdUrl="$someurl/$version/$filename"

      curl $createdUrl --output ./som/random/dir --create-dirs
      ...
    else
      version=$(jq -r --arg v keyvar $key 'streams.[$keyvar].branch' $vsconfig)
      branch=`jq --arg keyvar "streams.$key.branch" '$keyvar' $vsconfig`
      hash=`jq --arg keyvar "streams.$key.hash" '$keyvar' $vsconfig`

      filename = "$key-$version"
      if [ $branch == "some_branch" ]; then
        filename="one_file-$version-$branch-$hash.zip"
      else
        filename="$filename.zip"
      fi

      curl $createdUrl --output ./som/random/dir --create-dirs
    fi

    echo "Version: $version Branch: $branch Hash: $hash"
done

I've tried multiple formats, ie:

version=$(jq -r --arg v keyvar $key 'streams.[$keyvar].branch' $vsconfig)

And:

branch=`jq --arg keyvar "streams.$key.branch" '$keyvar' $vsconfig`

It gives this error:

jq: error: support/0 is not defined at <top-level>, line 1:

Answer 1

jq is quite sophisticated. The best bet is probably to do all of this inside a single jq invocation and have minimal or no bash scripting.

Values only

To start with, you can get each version/branch/hash object by applying [] to the .streams object. Using [] on an object—or key/value map—extracts the values and throws away the keys.

$ jq -c '.streams[]' vs.json
{"version":"2.33.0","branch":"ewewew","hash":"ewewewewe","widgets":[]}
{"version":"1.58.0","branch":"ewewew","hash":"ewewew","widgets":[]}
{"version":"1.42.0","branch":"ewewew","hash":"ewewe","widgets":[]}
{"version":"1.58.0","branch":"eewfwfef","hash":"vvfffsfsf","widgets":[]}
{"version":"1.39.0","branch":"fvrvvsdvds","hash":"vvsvdsvds","widgets":[]}

Then you can get the individual fields you're interested in by piping the above objects to a filter which grabs .version , .branch , and .hash and throws the three values into an array:

$ jq -c '.streams[] | [.version, .branch, .hash]' vs.json
["2.33.0","ewewew","ewewewewe"]
["1.58.0","ewewew","ewewew"]
["1.42.0","ewewew","ewewe"]
["1.58.0","eewfwfef","vvfffsfsf"]
["1.39.0","fvrvvsdvds","vvsvdsvds"]

To get it to format the results you can generate strings instead of lists and use \(...) to embed values. The -r flag tells it to print raw results: print the strings without quotes, in other words.

$ jq -r '.streams[] | "Version: \(.version) Branch: \(.branch) Hash: \(.hash)"' vs.json
Version: 2.33.0 Branch: ewewew Hash: ewewewewe
Version: 1.58.0 Branch: ewewew Hash: ewewew
Version: 1.42.0 Branch: ewewew Hash: ewewe
Version: 1.58.0 Branch: eewfwfef Hash: vvfffsfsf
Version: 1.39.0 Branch: fvrvvsdvds Hash: vvsvdsvds

Keys and values

To add the keys into the mix you can use to_entries , which extracts the key/value pairs from an object:

$ jq -c '.streams | to_entries[]' vs.json
{"key":"vs-first","value":{"version":"2.33.0","branch":"ewewew","hash":"ewewewewe","widgets":[]}}
{"key":"vs-second","value":{"version":"1.58.0","branch":"ewewew","hash":"ewewew","widgets":[]}}
{"key":"vs-third","value":{"version":"1.42.0","branch":"ewewew","hash":"ewewe","widgets":[]}}
{"key":"vs-fourth","value":{"version":"1.58.0","branch":"eewfwfef","hash":"vvfffsfsf","widgets":[]}}
{"key":"vs-fifth","value":{"version":"1.39.0","branch":"fvrvvsdvds","hash":"vvsvdsvds","widgets":[]}}

Pulling out the different fields then becomes:

$ jq -c '.streams | to_entries[] | [.key, .value.version, .value.branch, .value.hash]' vs.json
["vs-first","2.33.0","ewewew","ewewewewe"]
["vs-second","1.58.0","ewewew","ewewew"]
["vs-third","1.42.0","ewewew","ewewe"]
["vs-fourth","1.58.0","eewfwfef","vvfffsfsf"]
["vs-fifth","1.39.0","fvrvvsdvds","vvsvdsvds"]

Or equivalently, with the repeated .value lookups refactored out:

jq -c '.streams | to_entries[] | [.key, (.value | .version, .branch, .hash)]' vs.json
["vs-first","2.33.0","ewewew","ewewewewe"]
["vs-second","1.58.0","ewewew","ewewew"]
["vs-third","1.42.0","ewewew","ewewe"]
["vs-fourth","1.58.0","eewfwfef","vvfffsfsf"]
["vs-fifth","1.39.0","fvrvvsdvds","vvsvdsvds"]

Add bash processing

jq can't do everything, so if you do want to get the results out to bash to do additional processing—eg, call curl—you could use -r to print each value on a separate line and use read to read the lines into variables. It would look something like this:

jq -r '.streams | to_entries[] | .key, (.value | .version, .branch, .hash)' vs.json |
    while read -r key &&
          read -r version &&
          read -r branch &&
          read -r hash
    do
        ...
    done