堆棧內存溢出
  簡體   English   中英

使用jq用另一個json文件中的對象覆蓋json文件

[英]overwrite a json file with objects from another json file using jq

 原文  2017-10-30 19:29:35  4   2    json/ file/ merge/ jq

問題描述

改性：

您好，我在合並2個文件時遇到問題，基本上我有2個具有以下結構的json文件： 

[
  {
    "uri": "some/url.feature",
    "id": "safety-tests",
    "keyword": "Feature",
    "name": "Safety Tests",
    "description": "Some description",
    "line": 2,
    "tags": [
      {
        "name": "@sometag",
        "line": 1
      }
    ],
    "elements": [
      {
        "id": "some-element-id",
        "keyword": "Scenario Outline",
        "name": ": Some scenario name",
        "description": "",
        "line": 46,
        "type": "scenario",
        "tags": [
          {
            "name": "@sometag",
            "line": 1
          },
          {
            "name": "@someothertag",
            "line": 31
          }
        ],
        "before": [
          {
            "match": {
              "location": "some/test/file.rb:201"
            },
            "result": {
              "status": "passed",
              "duration": 15000
            }
          },
          {
            "match": {
              "location": "some/other/file.rb:5"
            },
            "result": {
              "status": "passed",
              "duration": 1722192000
            }
          }
        ],
        "steps": [
          {
            "keyword": "Given ",
            "name": "Some step name",
            "line": 46,
            "output": [
              "Some output"
            ],
            "match": {
              "location": "some/other/path/to/other/file.rb:137"
            },
            "result": {
              "status": "passed",
              "duration": 989158000
            }
          },
          {
            "keyword": "When ",
            "name": "some other step",
            "line": 46,
            "output": [
              "WARNING: static wait for 1 seconds."
            ],
            "match": {
              "location": "some/other/path/to/other/file.rb:80"
            },
            "result": {
              "status": "passed",
              "duration": 2700052000
            }
          },
          {
            "keyword": "And ",
            "name": "Some other name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:38"
            },
            "result": {
              "status": "passed",
              "duration": 954225000
            }
          },
          {
            "keyword": "Then ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:2"
            },
            "result": {
              "status": "passed",
              "duration": 38792000
            }
          },
          {
            "keyword": "And ",
            "name": "And again some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:2"
            },
            "result": {
              "status": "passed",
              "duration": 39268000
            }
          },
          {
            "keyword": "And ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:2"
            },
            "result": {
              "status": "passed",
              "duration": 55637000
            }
          },
          {
            "keyword": "And ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:2"
            },
            "result": {
              "status": "passed",
              "duration": 38375000
            }
          },
          {
            "keyword": "When ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:12"
            },
            "result": {
              "status": "passed",
              "duration": 751416000
            }
          },
          {
            "keyword": "And ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:2"
            },
            "result": {
              "status": "passed",
              "duration": 28043000
            }
          },
          {
            "keyword": "Then ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:20"
            },
            "result": {
              "status": "passed",
              "duration": 5204000
            }
          }
        ],
        "after": [
          {
            "match": {
              "location": "some/other/path/to/other/file.rb:91"
            },
            "result": {
              "status": "passed",
              "duration": 20000
            }
          },
          {
            "match": {
              "location": "some/other/path/to/other/file.rb:52"
            },
            "result": {
              "status": "passed",
              "duration": 5585000
            }
          },
          {
            "match": {
              "location": "some/other/path/to/other/file.rb:27"
            },
            "result": {
              "status": "passed",
              "duration": 168146000
            }
          },
          {
            "match": {
              "location": "some/other/path/to/other/file.rb:428"
            },
            "result": {
              "status": "passed",
              "duration": 62000
            }
          }
        ]
      },
      {
        "id": "some-element-id",
        "keyword": "Scenario Outline",
        "name": ": Some scenario name",
        "description": "",
        "line": 46,
        "type": "scenario",
        "tags": [
          {
            "name": "@sometag",
            "line": 1
          },
          {
            "name": "@someothertag",
            "line": 31
          }
        ],
        "before": [
          {
            "match": {
              "location": "some/test/file.rb:201"
            },
            "result": {
              "status": "passed",
              "duration": 15000
            }
          },
          {
            "match": {
              "location": "some/other/file.rb:5"
            },
            "result": {
              "status": "passed",
              "duration": 1722192000
            }
          }
        ],
        "steps": [
          {
            "keyword": "Given ",
            "name": "Some step name",
            "line": 46,
            "output": [
              "Some output"
            ],
            "match": {
              "location": "some/other/path/to/other/file.rb:137"
            },
            "result": {
              "status": "passed",
              "duration": 989158000
            }
          },
          {
            "keyword": "When ",
            "name": "some other step",
            "line": 46,
            "output": [
              "WARNING: static wait for 1 seconds."
            ],
            "match": {
              "location": "some/other/path/to/other/file.rb:80"
            },
            "result": {
              "status": "passed",
              "duration": 2700052000
            }
          },
          {
            "keyword": "And ",
            "name": "Some other name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:38"
            },
            "result": {
              "status": "passed",
              "duration": 954225000
            }
          },
          {
            "keyword": "Then ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:2"
            },
            "result": {
              "status": "passed",
              "duration": 38792000
            }
          },
          {
            "keyword": "And ",
            "name": "And again some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:2"
            },
            "result": {
              "status": "passed",
              "duration": 39268000
            }
          },
          {
            "keyword": "And ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:2"
            },
            "result": {
              "status": "passed",
              "duration": 55637000
            }
          },
          {
            "keyword": "And ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:2"
            },
            "result": {
              "status": "passed",
              "duration": 38375000
            }
          },
          {
            "keyword": "When ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:12"
            },
            "result": {
              "status": "passed",
              "duration": 751416000
            }
          },
          {
            "keyword": "And ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:2"
            },
            "result": {
              "status": "passed",
              "duration": 28043000
            }
          },
          {
            "keyword": "Then ",
            "name": "Some other step name",
            "line": 46,
            "match": {
              "location": "some/other/path/to/other/file.rb:20"
            },
            "result": {
              "status": "passed",
              "duration": 5204000
            }
          }
        ],
        "after": [
          {
            "match": {
              "location": "some/other/path/to/other/file.rb:91"
            },
            "result": {
              "status": "passed",
              "duration": 20000
            }
          },
          {
            "match": {
              "location": "some/other/path/to/other/file.rb:52"
            },
            "result": {
              "status": "passed",
              "duration": 5585000
            }
          },
          {
            "match": {
              "location": "some/other/path/to/other/file.rb:27"
            },
            "result": {
              "status": "passed",
              "duration": 168146000
            }
          },
          {
            "match": {
              "location": "some/other/path/to/other/file.rb:428"
            },
            "result": {
              "status": "passed",
              "duration": 62000
            }
          }
        ]
      }
    ]
  }
]

其中兩個文件中的elements可以包含任意數量的對象 。 這些是黃瓜的測試結果，因此文件A通常比文件B包含更多的元素，因為文件B是文件A中失敗測試的重新運行。

例如。 如果在第一遍中我們進行了100次測試，則文件A elements數組將包含100個具有上述格式的對象。 但是，如果在這100個測試中有50個失敗，則文件B elements數組將包含50個對象。 我要做的是用文件B覆蓋文件A elements數組，只是添加了在這兩個元素中重復的元素。 就像是

如果文件A有 

"elements":[{a:1, b:2, c:3, d:2, e:9, f:4}]

並且文件B有 

"elements":[{d:5}]

我想要新文件 

"elements":[{a:1, b:2, c:3, d:5, e:9, f:4}]

到目前為止，我已經 

jq '.[].elements' path/to/file/b > path/to/new/file
jq --argfile file path/to/new/file '.[].elements += $file' path/to/file/b

這使任何一起文件B中包含elements的內陣列elements在文件中的陣列，但不刪除它里面的復制對象。

我嘗試使用unique但對如何使用它一無所知。 有任何想法嗎？

經過幾次回應，我得到了 

jq --argfile b ~/Desktop/cucumber-rerun.json '.[0].elements[4] *= $b[0].elements[0]' ~/Desktop/cucumber.json

之所以可以正常工作，是因為在我的實際示例中，我知道文件A中的元素4是我要用文件B中的1唯一的元素覆蓋。但是，這對我不起作用，因為兩個文件都是自動生成的，並且順序的對象未知。

我想有一個命令，可以查看兩個文件並比較它們，並自動檢測A和B中的重復對象，然后用B中的對象覆蓋A中的對象

2 個解決方案

解決方案1
 0 已采納  2017-10-30 20:01:13

這是使用對象乘法的解決方案。 假設您的數據在A.json和B.json ： 

$ jq -M --argfile b B.json '.[0].elements[0] *= $b[0].elements[0]' A.json

產生 

[
  {
    "uri": "https://someurl.com",
    "id": "some-id",
    "keyword": "SomeKeyword",
    "name": "Some Name",
    "description": "Some description for that test result",
    "line": 2,
    "tags": [
      {
        "name": "@sometag",
        "line": 1
      }
    ],
    "elements": [
      {
        "a": 5,
        "b": 2
      }
    ]
  }
]

如果您的數組包含更多數據，則很容易推廣這種方法，但是您需要了解如何標識相應的元素。

關於修改后的問題，這是一個過濾器，該過濾B.json具有相同.id的A.json的對應對象更新A.json對象： 

def INDEX(stream; idx_expr):
  reduce stream as $row ({};
    .[$row|idx_expr| if type != "string" then tojson else . end] |= $row);

def merge_by_id(a;b):
  if b then INDEX(a[];.id) * INDEX(b[];.id) | map(.) else a end;

  INDEX($b[];.id) as $i
| map( .elements = merge_by_id(.elements; $i[.id].elements) )

例如，如果上述過濾器位於filter.jq ，則A.json包含修訂后的示例數據，而B.json包含 

[
  {
    "id": "safety-tests",
    "elements": [
      {
        "id": "some-element-id",
        "description": "updated description"
      }
    ]
  }
]

命令 

$ jq -M --argfile b B.json -f filter.jq A.json

產生結果 

[
  {
    "uri": "some/url.feature",
    "id": "safety-tests",                      <------ top level .id
    ...
    "elements": [
      {
        "id": "some-element-id",               <------ element .id
        "keyword": "Scenario Outline",
        "name": ": Some scenario name",
        "description": "updated description",  <------ updated value
        "line": 46,
        "type": "scenario",
        ...

請注意，以上解決方案假定A.json中元素的.id是唯一的，否則merge_by_id將不會產生所需的輸出。 在這種情況下，以下過濾器就足夠了： 

def INDEX(stream; idx_expr):
  reduce stream as $row ({};
    .[$row|idx_expr| if type != "string" then tojson else . end] |= $row);

  (INDEX($b[];.id) | map_values(INDEX(.elements[];.id))) as $i
| map( $i[.id] as $o | if $o then .elements |= map($o[.id]//.) else . end )

此過濾器僅要求B.json對象的.id唯一。 如果A.json和B.json中都可能存在非唯一元素，則需要更復雜的映射。

這是帶有注釋的過濾器版本： 

def INDEX(stream; idx_expr):
  reduce stream as $row ({};
    .[$row|idx_expr| if type != "string" then tojson else . end] |= $row);

  # first create a lookup table for elements from B.json
  (                                         #       [{id:x, elements:[{id:y, ...}]}]
      INDEX($b[];.id)                       # -> {x: {id:x, elements:[{id:y, ...}]}..}
    | map_values(INDEX(.elements[];.id))    # -> {x: {y: {id:y, ...}}}
  ) as $i

  # update A.json objects
| map(                                      # for each object in A.json
    $i[.id] as $o                           # do we have updated values from B.json ?
  | if $o then .elements |= map($o[.id]//.) # if so then replace corresponding elements
    else . end                              # otherwise leave object unchanged
  )

解決方案2
 0  2017-10-30 20:06:17

要簡潔地解決問題，需要兩個關鍵的見解：

  • 如果A和B是兩個對象，則可以將其與優先級結合為B，可以這樣寫：A + B

  • 要“就地”更新對象，我們可以根據需要使用| =或+ =。

對於您而言，我們可以通過以下方式應用這些見解： 

.[0].elements[0] += $B[0].elements[0]

假設遵循以下方式進行調用： 

jq --argfile B B.json -f combine.jq A.json

輸出量

使用您的輸入，輸出包括： 

"elements": [
  {
    "a": 5,
    "b": 2
  }
]

按照要求。

進一步說明

要用上述jq調用產生的輸出覆蓋A.json，您可能需要使用sponge ，但是請注意風險。

如果出於某種原因不想使用--argfile ，則可以改用--slurpfile ，但是必須寫$B[0][0]

暫無
暫無

聲明:本站的技術帖子網頁，遵循CC BY-SA 4.0協議，如果您需要轉載，請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.

相關問題 使用jq將json對象從一個文件添加到單個數組下的另一個文件 使用 jq 使用另一個文件中的值替換 JSON 中的值 在jq中從另一個減去一個json文件 使用jq將一個json文件加載到另一個json文件的哈希中 使用“jq”從另一個 json 文件附加到 json 文件中的數組 使用 jq 從 json 文件生成模式 使用JQ從json文件中提取信息 使用 jq 從 json 文件中刪除元素 如何通過使用Jq將JSON文件的對象轉換為對象數組？ 使用JQ使用來自另一個JSON的值更新一個JSON文件值
相關標簽
 
粵ICP備18138465號  © 2020-2024 STACKOOM.COM