Setting a value in jq when path and value are env variables is not working as expected (bash)

Question

I want to set a value in a JSON nested object using jq from a shell script. If I use explicit values, it sets the JSON value as expected. But when I use variables, it adds (if strangely) a new key/value instead.

The end goal is to set a key's value in arbitrary JSON given an arbitrary, jp-appropriate, explicit path. No selecting/matching required. Let's also assume all values are Strings. I get diff between --args and --argsjson.

But let's start with the simple. Here is my test:

echo $JSON
{
    "make" : "ford",
    "model" : "ranger",
    "year" : "2020"
    "options" : {
        "cylinders" : "v6"
        "cab" : "true"
    }
}

Let's say I want to set options.cylinders = "v4"
This works:

echo $JSON | jq 'options.cylinders = "v4"'

This works as well:

echo $JSON | jq --arg value "v4" '.options.cylinders = $value'

Let's replace value with variable. This works.

cyl="v4"
echo $JSON | jq --arg value "$cyl" '.options.cylinders = $value'

Let's replace the path with variable. It complains, but just to let you know where I've been:

cyl="v4"
path="options.cylinder"
echo $JSON | jq --arg path "$path" --arg value "$cyl" '.$path = $value'

That throws error, but it is corrected by:

cyl="v4"
path="options.cylinder"
echo $JSON | jq --arg path "$path" --arg value "$cyl" '.[$path] = $value'

Now you would think the former would work. It succeeds but it adds the value with the entire path as key, instead of setting it as if it where the path. The result is:

{
    "make" : "ford",
    "model" : "ranger",
    "year" : "2020"
    "options" : {
        "cylinders" : "v6"
        "cab" : "true"
    }
    "options.cylinders" : "v4"
}

I have also tried using the path() function, passing it an bash array. This does not work either.

cyl="v4"
path=(".", "options", "cylinders")
echo "${JSON}" | jq --arg key $path --arg value "${cyl}" 'path($key; $value)'

I get:

jq: error: path/2 is not defined at <top-level>, line 1:
path($key; $value)
jq: 1 compile error

If I remove the "." from the path, it just moves up the error:

jq: error: path/1 is not defined at <top-level>, line 1:

I have read all the doc. I am sure I have tried other things, but I'm stumped at this point. So...what am I missing here? Close? Ideas?

Answer 1

Credits to @pmf so making it a wiki

#!/bin/sh

# Sets the source JSON in a shell variable
json='{
  "make": "ford",
  "model": "ranger",
  "year": "2020",
  "options": {
    "cylinders": "v6",
    "cab": "true"
  }
}'

# Parameters to modify the source JSON
path='options.cylinders'
cyl='v4'

# Process with jq
jq \
  --null-input \
  --argjson json "$json" \
  --arg path "$path" \
  --arg value "$cyl" \
  '$json | setpath($path / "."; $value)'

setpath(path_array; value) sets the value at path with path elements as an array.

So the $path string is: / divided by its . dots, to turn it into an array.

Lets demo this:

jq -n '"options.cylinders" / "."'

turns the string above into an array suitable for creating a path from the dot delimited string:

[
  "options",
  "cylinders"
]

So when it runs:

setpath("options.cylinders" / "."; "v4")

it expands into:

setpath(["options","cylinders"]; "v4")

which is exactly the same as static path value assignment:

.options.cylinders = "v4"