jq json object concatenation to bash string array

Question

I want to use jq (or anything else when it's the wrong tool) to concatenate a json object like this:

{
  "https://github.com": {
    "user-one": {
      "repository-one": "version-one",
      "repository-two": "version-two"
    },
    "user-two": {
      "repository-three": "version-three",
      "repository-four": "version-four"
    }
  },
  "https://gitlab.com": {
    "user-three": {
      "repository-five": "version-five",
      "repository-six": "version-six"
    },
    "user-four": {
      "repository-seven": "version-seven",
      "repository-eight": "version-eight"
    }
  }
}

recursively to a bash string array like this:

(
    "https://github.com/user-one/repository-one/archive/refs/heads/version-one.tar.gz"
    "https://github.com/user-one/repository-two/archive/refs/heads/version-two.tar.gz"
    "https://github.com/user-two/repository-three/archive/refs/heads/version-three.tar.gz"
    "https://github.com/user-two/repository-four/archive/refs/heads/version-four.tar.gz"
    "https://gitlab.com/user-three/repository-five/-/archive/version-five/repository-five-version-five.tar.gz"
    "https://gitlab.com/user-three/repository-six/-/archive/version-six/repository-six-version-six.tar.gz"
    "https://gitlab.com/user-four/repository-seven/-/archive/version-seven/repository-seven-version-seven.tar.gz"
    "https://gitlab.com/user-four/repository-eight/-/archive/version-eight/repository-eight-version-eight.tar.gz"
)

for subsequent use in a loop.

for i in "${arr[@]}"
do
   echo "$i"
done

Have no idea how to do that. As you can see, the values must be handled differently depending on the object name.

"https://github.com" + "/" + $user_name + "/" + $repository_name + "/archive/refs/heads/" + $version + ".tar.gz"

"https://gitlab.com" + "/" + $user_name + "/" + $repository_name + "/-/archive/" + $version + "/" + $repository_name + "-" + $version + ".tar.gz"

Could anyone help?

Answer 1

Easily done.

First, let's focus on the jq code alone:

to_entries[]                # split items into keys and values
| .key as $site             # store first key in $site
| .value                    # code below deals with the value
| to_entries[]              # split that value into keys and values
| .key as $user             # store the key in $user
| .value                    # code below deals with the value
| to_entries[]              # split that value into keys and values
| .key as $repository_name  # store the key in $repository_name
| .value as $version        # store the value in $version
| if $site == "https://github.com" then
    "\($site)/\($user)/\($repository_name)/archive/refs/heads/\($version).tar.gz"
  else
    "\($site)/\($user)/\($repository_name)/-/archive/\($version)/\($repository_name)-\($version).tar.gz"
  end

That generates a list of lines. Reading lines into a bash array looks like readarray -t arrayname <...datasource...

Thus, using a process substitution to redirect jq's stdout as if it were a file:

readarray -t uris < <(jq -r '
  to_entries[]
  | .key as $site
  | .value
  | to_entries[]
  | .key as $user
  | .value
  | to_entries[]
  | .key as $repository_name
  | .value as $version
  | if $site == "https://github.com" then
      "\($site)/\($user)/\($repository_name)/archive/refs/heads/\($version).tar.gz"
    else
      "\($site)/\($user)/\($repository_name)/-/archive/\($version)/\($repository_name)-\($version).tar.gz"
    end
  ' <config.json
)

Answer 2

The basic task of generating the strings can be done efficiently and generically (ie, without any limits on the depths of the basenames) using the jq filter:

paths(strings) as $p | $p + [getpath($p)] | join("/")

There are several ways to populate a bash array accordingly, but if you merely wish to iterate through the values, you could use a bash while loop, like so:

< input.json jq -r '
  paths(strings) as $p | $p + [getpath($p)] | join("/")' | 
  while read -r line ; do
    echo "$line"
  done

You might also wish to consider using jq's @sh or @uri filter. For a jq urlencode function, see eg https://rosettacode.org/wiki/URL_encoding#jq

(If the strings contain newlines or tabs, then the above would need to be tweaked accordingly.)