使用 JQ（在所有级别上）使用来自另一个 JSON 的值更新一个 JSON 文件值

Question

I have two JSON files:

source.json:

{
  "general": {
    "level1": {
      "key1": "x-x-x-x-x-x-x-x",
      "key3": "z-z-z-z-z-z-z-z",
      "key4": "w-w-w-w-w-w-w-w"
    },
    "another" : {
      "key": "123456",
      "comments": {
        "one": "111",
        "other": "222"
      }
    }
  },
  "title": "The best"
}

and the

target.json:

{
  "general": {
    "level1": {
      "key1": "xxxxxxxx",
      "key2": "yyyyyyyy",
      "key3": "zzzzzzzz"
    },
    "onemore": {
      "kkeeyy": "0000000"
    }
  },
  "specific": {
    "stuff": "test"
  },
  "title": {
    "one": "one title",
    "other": "other title"
  }
}

I need all the values for keys which exist in both files, copied from source.json to target.json , considering all the levels.
I've seen and tested the solution from this post . It only copies the first level of keys, and I couldn't get it to do what I need. The result from solution in this post , looks like this:

{
  "general": {
    "level1": {
      "key1": "x-x-x-x-x-x-x-x",
      "key3": "z-z-z-z-z-z-z-z",
      "key4": "w-w-w-w-w-w-w-w"
    },
    "another": {
      "key": "123456",
      "comments": {
        "one": "111",
        "other": "222"
      }
    }
  },
  "specific": {
    "stuff": "test"
  },
  "title": "The best"
}

Everything under the "general" key was copied as is.
What I need, is this:

{
  "general": {
    "level1": {
      "key1": "x-x-x-x-x-x-x-x",
      "key2": "yyyyyyyy",
      "key3": "z-z-z-z-z-z-z-z"
    },
    "onemore": {
      "kkeeyy": "0000000"
    }
  },
  "specific": {
    "stuff": "test"
  },
  "title": {
    "one": "one title",
    "other": "other title"
  }
}

Only "key1" and "key3" should be copied.
Keys in target JSON must not be deleted and new keys should not be created.

Can anyone help?

Answer 1

One approach you could take is get all the paths to all scalar values for each input and take the set intersections. Then copy values from source to target from those paths.

First we'll need an intersect function (which was surprisingly difficult to craft):

def set_intersect($other):
    (map({ ($other[] | tojson): true }) | add) as $o
    | reduce (.[] | tojson) as $v ({}; if $o[$v] then .[$v] = true else . end)
    | keys_unsorted
    | map(fromjson);

Then to do the update:

$ jq --argfile s source.json '
reduce ([paths(scalars)] | set_intersect([$s | paths(scalars)])[]) as $p (.;
    setpath($p; $s | getpath($p))
)
' target.json

Answer 2

[Note: this response answers the original question, with respect to the original data. The OP may have had paths in mind rather than keys.]

There is no need to compute the intersection to achieve a reasonably efficient solution.

First, let's hypothesize the following invocation of jq:

jq -n --argfile source source.json --argfile target target.json -f copy.jq

In the file copy.jq, we can begin by defining a helper function:

# emit an array of the distinct terminal keys in the input entity
def keys: [paths | .[-1] | select(type=="string")] | unique;

In order to inspect all the paths to leaf elements of $source , we can use tostream :

($target | keys) as $t
| reduce ($source|tostream|select(length==2)) as [$p,$v]
    ($target;
     if $t|index($p[-1]) then setpath($p; $v) else . end)

Alternatives

Since $t is sorted, it would (at least in theory) make sense to use bsearch instead of index :

 bsearch($p[-1]) > -1

Also, instead of tostream we could use paths(scalars) .

Putting these alternatives together:

($target | keys) as $t
| reduce ($source|paths(scalars)) as $p
    ($target;
     if $t|bsearch($p[-1]) > -1 
     then setpath($p; $source|getpath($p))
     else . end)

Output

{
  "general": {
    "level1": {
      "key1": "x-x-x-x-x-x-x-x",
      "key2": "yyyyyyyy",
      "key3": "z-z-z-z-z-z-z-z"
    },
    "onemore": {
      "kkeeyy": "0000000"
    }
  },
  "specific": {
    "stuff": "test"
  }
}

Answer 3

The following provides a solution to the revised question, which is actually about "paths" rather than "keys".

([$target|paths(scalars)] | unique) as $paths
| reduce ($source|paths(scalars)) as $p
    ($target;
     if $paths | bsearch($p) > -1 
     then setpath($p; $source|getpath($p))
     else . end)

unique is called so that binary search can be used subsequently.

Invocation:

jq -n --argfile source source.json --argfile target target.json -f program.jq