堆栈内存溢出
  繁体   English   中英

使用jq用另一个json文件中的对象覆盖json文件

[英]overwrite a json file with objects from another json file using jq

 原文  2017-10-30 19:29:35  0   2    json/ file/ merge/ jq

问题描述

改性：

您好，我在合并2个文件时遇到问题，基本上我有2个具有以下结构的json文件： 

[
  {
    "uri": "some/url.feature",
    "id": "safety-tests",
    "keyword": "Feature",
    "name": "Safety Tests",
    "description": "Some description",
    "line": 2,
    "tags": [
      {
        "name": "@sometag",
        "line": 1
      }
    ],
    "elements": [
      {
        "id": "some-element-id",
        "keyword": "Scenario Outline",
        "name": ": Some scenario name",
        "description": "",
        "line": 46,
        "type": "scenario",
        "tags": [
          {
            "name": "@sometag",
            "line": 1
          },
          {
            "name": "@someothertag",
            "line": 31
          }
        ],
        "before": [
          {
            "match": {
              "location": "some/test/file.rb:201"
            },
            "result": {
              "status": "passed",
              "duration": 15000
            }
          },
          {
            "match": {
              "location": "some/other/file.rb:5"
            },
            "result": {
              "status": "passed",
              "duration": 1722192000
            }
          }
        ],
        "steps": [
          {
            "keyword": "Given ",
            "name": "Some step name",
            "line": 46,
            "output": [
              "Some output"
            ],
            "match": {
              "location": "some/other/path/to/other/file.rb:137"
            },
            "result": {
              "status": "passed",
              "duration": 989158000
            }
          },
          {
            "keyword": "When ",
            "name": "some other step",
            "line": 46,
            "output": [
              "WARNING: static wait for 1 seconds."
            ],
            "match": {
              "location": "some/other/path/to/other/file.rb:80"
            },
            "result": {
              "status": "passed",
              "duration": 2700052000
            }
          },
          {
            "keyword": "And ",
            "name": "Some other name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:38"
            },
            "result": {
              "status": "passed",
              "duration": 954225000
            }
          },
          {
            "keyword": "Then ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:2"
            },
            "result": {
              "status": "passed",
              "duration": 38792000
            }
          },
          {
            "keyword": "And ",
            "name": "And again some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:2"
            },
            "result": {
              "status": "passed",
              "duration": 39268000
            }
          },
          {
            "keyword": "And ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:2"
            },
            "result": {
              "status": "passed",
              "duration": 55637000
            }
          },
          {
            "keyword": "And ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:2"
            },
            "result": {
              "status": "passed",
              "duration": 38375000
            }
          },
          {
            "keyword": "When ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:12"
            },
            "result": {
              "status": "passed",
              "duration": 751416000
            }
          },
          {
            "keyword": "And ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:2"
            },
            "result": {
              "status": "passed",
              "duration": 28043000
            }
          },
          {
            "keyword": "Then ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:20"
            },
            "result": {
              "status": "passed",
              "duration": 5204000
            }
          }
        ],
        "after": [
          {
            "match": {
              "location": "some/other/path/to/other/file.rb:91"
            },
            "result": {
              "status": "passed",
              "duration": 20000
            }
          },
          {
            "match": {
              "location": "some/other/path/to/other/file.rb:52"
            },
            "result": {
              "status": "passed",
              "duration": 5585000
            }
          },
          {
            "match": {
              "location": "some/other/path/to/other/file.rb:27"
            },
            "result": {
              "status": "passed",
              "duration": 168146000
            }
          },
          {
            "match": {
              "location": "some/other/path/to/other/file.rb:428"
            },
            "result": {
              "status": "passed",
              "duration": 62000
            }
          }
        ]
      },
      {
        "id": "some-element-id",
        "keyword": "Scenario Outline",
        "name": ": Some scenario name",
        "description": "",
        "line": 46,
        "type": "scenario",
        "tags": [
          {
            "name": "@sometag",
            "line": 1
          },
          {
            "name": "@someothertag",
            "line": 31
          }
        ],
        "before": [
          {
            "match": {
              "location": "some/test/file.rb:201"
            },
            "result": {
              "status": "passed",
              "duration": 15000
            }
          },
          {
            "match": {
              "location": "some/other/file.rb:5"
            },
            "result": {
              "status": "passed",
              "duration": 1722192000
            }
          }
        ],
        "steps": [
          {
            "keyword": "Given ",
            "name": "Some step name",
            "line": 46,
            "output": [
              "Some output"
            ],
            "match": {
              "location": "some/other/path/to/other/file.rb:137"
            },
            "result": {
              "status": "passed",
              "duration": 989158000
            }
          },
          {
            "keyword": "When ",
            "name": "some other step",
            "line": 46,
            "output": [
              "WARNING: static wait for 1 seconds."
            ],
            "match": {
              "location": "some/other/path/to/other/file.rb:80"
            },
            "result": {
              "status": "passed",
              "duration": 2700052000
            }
          },
          {
            "keyword": "And ",
            "name": "Some other name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:38"
            },
            "result": {
              "status": "passed",
              "duration": 954225000
            }
          },
          {
            "keyword": "Then ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:2"
            },
            "result": {
              "status": "passed",
              "duration": 38792000
            }
          },
          {
            "keyword": "And ",
            "name": "And again some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:2"
            },
            "result": {
              "status": "passed",
              "duration": 39268000
            }
          },
          {
            "keyword": "And ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:2"
            },
            "result": {
              "status": "passed",
              "duration": 55637000
            }
          },
          {
            "keyword": "And ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:2"
            },
            "result": {
              "status": "passed",
              "duration": 38375000
            }
          },
          {
            "keyword": "When ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:12"
            },
            "result": {
              "status": "passed",
              "duration": 751416000
            }
          },
          {
            "keyword": "And ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:2"
            },
            "result": {
              "status": "passed",
              "duration": 28043000
            }
          },
          {
            "keyword": "Then ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:20"
            },
            "result": {
              "status": "passed",
              "duration": 5204000
            }
          }
        ],
        "after": [
          {
            "match": {
              "location": "some/other/path/to/other/file.rb:91"
            },
            "result": {
              "status": "passed",
              "duration": 20000
            }
          },
          {
            "match": {
              "location": "some/other/path/to/other/file.rb:52"
            },
            "result": {
              "status": "passed",
              "duration": 5585000
            }
          },
          {
            "match": {
              "location": "some/other/path/to/other/file.rb:27"
            },
            "result": {
              "status": "passed",
              "duration": 168146000
            }
          },
          {
            "match": {
              "location": "some/other/path/to/other/file.rb:428"
            },
            "result": {
              "status": "passed",
              "duration": 62000
            }
          }
        ]
      }
    ]
  }
]

其中两个文件中的elements可以包含任意数量的对象 。 这些是黄瓜的测试结果，因此文件A通常比文件B包含更多的元素，因为文件B是文件A中失败测试的重新运行。

例如。 如果在第一遍中我们进行了100次测试，则文件A elements数组将包含100个具有上述格式的对象。 但是，如果在这100个测试中有50个失败，则文件B elements数组将包含50个对象。 我要做的是用文件B覆盖文件A elements数组，只是添加了在这两个元素中重复的元素。 就像是

如果文件A有 

"elements":[{a:1, b:2, c:3, d:2, e:9, f:4}]

并且文件B有 

"elements":[{d:5}]

我想要新文件 

"elements":[{a:1, b:2, c:3, d:5, e:9, f:4}]

到目前为止，我已经 

jq '.[].elements' path/to/file/b > path/to/new/file
jq --argfile file path/to/new/file '.[].elements += $file' path/to/file/b

这使任何一起文件B中包含elements的内阵列elements在文件中的阵列，但不删除它里面的复制对象。

我尝试使用unique但对如何使用它一无所知。 有任何想法吗？

经过几次回应，我得到了 

jq --argfile b ~/Desktop/cucumber-rerun.json '.[0].elements[4] *= $b[0].elements[0]' ~/Desktop/cucumber.json

之所以可以正常工作，是因为在我的实际示例中，我知道文件A中的元素4是我要用文件B中的1唯一的元素覆盖。但是，这对我不起作用，因为两个文件都是自动生成的，并且顺序的对象未知。

我想有一个命令，可以查看两个文件并比较它们，并自动检测A和B中的重复对象，然后用B中的对象覆盖A中的对象

2 个解决方案

解决方案1
 0 已采纳  2017-10-30 20:01:13

这是使用对象乘法的解决方案。 假设您的数据在A.json和B.json ： 

$ jq -M --argfile b B.json '.[0].elements[0] *= $b[0].elements[0]' A.json

产生 

[
  {
    "uri": "https://someurl.com",
    "id": "some-id",
    "keyword": "SomeKeyword",
    "name": "Some Name",
    "description": "Some description for that test result",
    "line": 2,
    "tags": [
      {
        "name": "@sometag",
        "line": 1
      }
    ],
    "elements": [
      {
        "a": 5,
        "b": 2
      }
    ]
  }
]

如果您的数组包含更多数据，则很容易推广这种方法，但是您需要了解如何标识相应的元素。

关于修改后的问题，这是一个过滤器，该过滤B.json具有相同.id的A.json的对应对象更新A.json对象： 

def INDEX(stream; idx_expr):
  reduce stream as $row ({};
    .[$row|idx_expr| if type != "string" then tojson else . end] |= $row);

def merge_by_id(a;b):
  if b then INDEX(a[];.id) * INDEX(b[];.id) | map(.) else a end;

  INDEX($b[];.id) as $i
| map( .elements = merge_by_id(.elements; $i[.id].elements) )

例如，如果上述过滤器位于filter.jq ，则A.json包含修订后的示例数据，而B.json包含 

[
  {
    "id": "safety-tests",
    "elements": [
      {
        "id": "some-element-id",
        "description": "updated description"
      }
    ]
  }
]

命令 

$ jq -M --argfile b B.json -f filter.jq A.json

产生结果 

[
  {
    "uri": "some/url.feature",
    "id": "safety-tests",                      <------ top level .id
    ...
    "elements": [
      {
        "id": "some-element-id",               <------ element .id
        "keyword": "Scenario Outline",
        "name": ": Some scenario name",
        "description": "updated description",  <------ updated value
        "line": 46,
        "type": "scenario",
        ...

请注意，以上解决方案假定A.json中元素的.id是唯一的，否则merge_by_id将不会产生所需的输出。 在这种情况下，以下过滤器就足够了： 

def INDEX(stream; idx_expr):
  reduce stream as $row ({};
    .[$row|idx_expr| if type != "string" then tojson else . end] |= $row);

  (INDEX($b[];.id) | map_values(INDEX(.elements[];.id))) as $i
| map( $i[.id] as $o | if $o then .elements |= map($o[.id]//.) else . end )

此过滤器仅要求B.json对象的.id唯一。 如果A.json和B.json中都可能存在非唯一元素，则需要更复杂的映射。

这是带有注释的过滤器版本： 

def INDEX(stream; idx_expr):
  reduce stream as $row ({};
    .[$row|idx_expr| if type != "string" then tojson else . end] |= $row);

  # first create a lookup table for elements from B.json
  (                                         #       [{id:x, elements:[{id:y, ...}]}]
      INDEX($b[];.id)                       # -> {x: {id:x, elements:[{id:y, ...}]}..}
    | map_values(INDEX(.elements[];.id))    # -> {x: {y: {id:y, ...}}}
  ) as $i

  # update A.json objects
| map(                                      # for each object in A.json
    $i[.id] as $o                           # do we have updated values from B.json ?
  | if $o then .elements |= map($o[.id]//.) # if so then replace corresponding elements
    else . end                              # otherwise leave object unchanged
  )

解决方案2
 0  2017-10-30 20:06:17

要简洁地解决问题，需要两个关键的见解：

  • 如果A和B是两个对象，则可以将其与优先级结合为B，可以这样写：A + B

  • 要“就地”更新对象，我们可以根据需要使用| =或+ =。

对于您而言，我们可以通过以下方式应用这些见解： 

.[0].elements[0] += $B[0].elements[0]

假设遵循以下方式进行调用： 

jq --argfile B B.json -f combine.jq A.json

输出量

使用您的输入，输出包括： 

"elements": [
  {
    "a": 5,
    "b": 2
  }
]

按照要求。

进一步说明

要用上述jq调用产生的输出覆盖A.json，您可能需要使用sponge ，但是请注意风险。

如果出于某种原因不想使用--argfile ，则可以改用--slurpfile ，但是必须写$B[0][0]

暂无
暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 使用jq将json对象从一个文件添加到单个数组下的另一个文件 使用 jq 使用另一个文件中的值替换 JSON 中的值 在jq中从另一个减去一个json文件 使用jq将一个json文件加载到另一个json文件的哈希中 使用“jq”从另一个 json 文件附加到 json 文件中的数组 使用 jq 从 json 文件生成模式 使用JQ从json文件中提取信息 使用 jq 从 json 文件中删除元素 如何通过使用Jq将JSON文件的对象转换为对象数组？ 使用JQ使用来自另一个JSON的值更新一个JSON文件值
相关标签
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM