堆棧內存溢出
  簡體   English   中英

從 jq 導出結果到 csv

[英]export results from jq to csv

 原文  2021-12-22 18:08:32  3   2    json/ csv/ jq

問題描述

我有以下數據集，我想將其導出到 CSV 中：

數據集：

{
  "data": {
    "activeFindings": {
      "findings": [
        {
          "findingId": "risk#80703",
          "accountId": "00000000-000000-0000000-000000",
          "products": [
            "GWSERVER01"
          ],
          "findingDisplayName": "risk#80703",
          "severity": "CRITICAL",
          "findingDescription": "PSOD with re-formatting a valid dedup metadata block.",
          "findingImpact": "Potential ESXi host crash",
          "recommendations": [
            "This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523)",
            "This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804)"
          ],
          "kbLinkURLs": [
            "https://kb.vmware.com/s/article/80703"
          ],
          "recommendationsVCF": [
            "This issue is resolved with VMware Cloud Foundation 4.1"
          ],
          "kbLinkURLsVCF": [
            "https://kb.vmware.com/s/article/80703"
          ],
          "categoryName": "Storage",
          "findingTypes": [
            "UPGRADE"
          ],
          "firstObserved": 1629806351877,
          "totalAffectedObjectsCount": 12,
          "affectedObjects": [
            {
              "sourceName": "GWSERVER01.corp.contoso.org",
              "objectName": "server01.corp.contoso.org",
              "objectType": "ESX",
              "version": "6.7.0",
              "buildNumber": "17499825",
              "solutionTags": [],
              "firstObserved": 1629806351877
            },
            {
              "sourceName": "GWSERVER01.corp.contoso.org",
              "objectName": "server02.corp.contoso.org",
              "objectType": "ESX",
              "version": "6.7.0",
              "buildNumber": "17499825",
              "solutionTags": [],
              "firstObserved": 1629806351877
            },
            {
              "sourceName": "GWSERVER01.corp.contoso.org",
              "objectName": "server03.corp.contoso.org",
              "objectType": "ESX",
              "version": "6.7.0",
              "buildNumber": "17499825",
              "solutionTags": [],
              "firstObserved": 1629806351877
            },
            {
              "sourceName": "GWSERVER01.corp.contoso.org",
              "objectName": "server04.corp.contoso.org",
              "objectType": "ESX",
              "version": "6.7.0",
              "buildNumber": "17499825",
              "solutionTags": [],
              "firstObserved": 1629806351877
            },
            {
              "sourceName": "GWSERVER01.corp.contoso.org",
              "objectName": "server05.corp.contoso.org",
              "objectType": "ESX",
              "version": "6.7.0",
              "buildNumber": "17499825",
              "solutionTags": [],
              "firstObserved": 1629806351877
            },
            {
              "sourceName": "GWSERVER01.corp.contoso.org",
              "objectName": "server06.corp.contoso.org",
              "objectType": "ESX",
              "version": "6.7.0",
              "buildNumber": "17499825",
              "solutionTags": [],
              "firstObserved": 1629806351877
            },
            {
              "sourceName": "GWSERVER01.corp.contoso.org",
              "objectName": "server07.corp.contoso.org",
              "objectType": "ESX",
              "version": "6.7.0",
              "buildNumber": "17499825",
              "solutionTags": [],
              "firstObserved": 1629806351877
            }
          ]
        }
      ],
      "totalRecords": 1,
      "timeTaken": 56
    }
  }
}
{
  "data": {
    "activeFindings": {
      "findings": [
        {
          "findingId": "risk#80703",
          "accountId": "00000000-000000-0000000-000000",
          "products": [
            "GWSERVER02.corp.contoso.org"
          ],
          "findingDisplayName": "risk#80703",
          "severity": "CRITICAL",
          "findingDescription": "PSOD with re-formatting a valid dedup metadata block.",
          "findingImpact": "Potential ESXi host crash",
          "recommendations": [
            "This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523)",
            "This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804)"
          ],
          "kbLinkURLs": [
            "https://kb.vmware.com/s/article/80703"
          ],
          "recommendationsVCF": [
            "This issue is resolved with VMware Cloud Foundation 4.1"
          ],
          "kbLinkURLsVCF": [
            "https://kb.vmware.com/s/article/80703"
          ],
          "categoryName": "Storage",
          "findingTypes": [
            "UPGRADE"
          ],
          "firstObserved": 1635968448112,
          "totalAffectedObjectsCount": 2,
          "affectedObjects": [
            {
              "sourceName": "GWSERVER02.corp.contoso.org",
              "objectName": "server10.corp.contoso.org",
              "objectType": "ESX",
              "version": "6.7.0",
              "buildNumber": "17167734",
              "solutionTags": [],
              "firstObserved": 1635968448112
            },
            {
              "sourceName": "GWSERVER02.corp.contoso.org",
              "objectName": "server11.corp.contoso.org",
              "objectType": "ESX",
              "version": "6.7.0",
              "buildNumber": "17167734",
              "solutionTags": [],
              "firstObserved": 1635968448112
            }
          ]
        }
      ],
      "totalRecords": 1,
      "timeTaken": 51
    }
  }
}

header 如下：

"Finding Id","Issue Description","Risk if no Action Taken","Severity","Recommendations","Source Name","Object Name","Object Type","Host Version","Build","First Observed","Reference"

Header 按鍵映射如下：

  • 查找 ID = 查找 ID
  • 問題描述 = 發現描述
  • 不采取行動的風險 = 發現影響
  • 嚴重性 = 嚴重性
  • 推薦=推薦
  • 源名稱 = 源名稱
  • Object 名稱 = 對象名稱
  • Object 類型 = objectType
  • 主機版本 = 版本
  • 構建 = buildNumber
  • 第一個觀察到 = 第一個觀察到
  • 參考 = kbLinkURLs

不幸的是，我們必須對每個發現和產品執行 API 調用（例如：我們無法一次提取所有產品的所有發現 - API 不允許我們執行此類查詢，因此，我們必須進行多次調用以獲取所有結果及其相關對象。）

話雖如此，將數據導出到 csv 的首選方法是什么？ 盡管我們必須遍歷多個節點，但使用 jq 的 @CSV 會起作用嗎？

任何幫助/指導將不勝感激。

謝謝！

注 1： chepner要求的數據集的剝離版本

{
  "data": {
    "activeFindings": {
      "findings": [
        {
          "findingId": "risk#80703",
          "accountId": "00000000-000000-0000000-000000",
          "products": [
            "GWSERVER01"
          ],
          "findingDisplayName": "risk#80703",
          "severity": "CRITICAL",
          "findingDescription": "PSOD with re-formatting a valid dedup metadata block.",
          "findingImpact": "Potential ESXi host crash",
          "recommendations": [
            "This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523)",
            "This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804)"
          ],
          "kbLinkURLs": [
            "https://kb.vmware.com/s/article/80703"
          ],
          "recommendationsVCF": [
            "This issue is resolved with VMware Cloud Foundation 4.1"
          ],
          "kbLinkURLsVCF": [
            "https://kb.vmware.com/s/article/80703"
          ],
          "categoryName": "Storage",
          "findingTypes": [
            "UPGRADE"
          ],
          "firstObserved": 1629806351877,
          "totalAffectedObjectsCount": 12,
          "affectedObjects": [
            {
              "sourceName": "GWSERVER01.corp.contoso.org",
              "objectName": "server01.corp.contoso.org",
              "objectType": "ESX",
              "version": "6.7.0",
              "buildNumber": "17499825",
              "solutionTags": [],
              "firstObserved": 1629806351877
            },
            {
              "sourceName": "GWSERVER01.corp.contoso.org",
              "objectName": "server02.corp.contoso.org",
              "objectType": "ESX",
              "version": "6.7.0",
              "buildNumber": "17499825",
              "solutionTags": [],
              "firstObserved": 1629806351877
            },
          ]
        }
      ],
      "totalRecords": 1,
      "timeTaken": 56
    }
  }
}
{
  "data": {
    "activeFindings": {
      "findings": [
        {
          "findingId": "risk#80703",
          "accountId": "00000000-000000-0000000-000000",
          "products": [
            "GWSERVER02.corp.contoso.org"
          ],
          "findingDisplayName": "risk#80703",
          "severity": "CRITICAL",
          "findingDescription": "PSOD with re-formatting a valid dedup metadata block.",
          "findingImpact": "Potential ESXi host crash",
          "recommendations": [
            "This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523)",
            "This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804)"
          ],
          "kbLinkURLs": [
            "https://kb.vmware.com/s/article/80703"
          ],
          "recommendationsVCF": [
            "This issue is resolved with VMware Cloud Foundation 4.1"
          ],
          "kbLinkURLsVCF": [
            "https://kb.vmware.com/s/article/80703"
          ],
          "categoryName": "Storage",
          "findingTypes": [
            "UPGRADE"
          ],
          "firstObserved": 1635968448112,
          "totalAffectedObjectsCount": 2,
          "affectedObjects": [
            {
              "sourceName": "GWSERVER02.corp.contoso.org",
              "objectName": "server10.corp.contoso.org",
              "objectType": "ESX",
              "version": "6.7.0",
              "buildNumber": "17167734",
              "solutionTags": [],
              "firstObserved": 1635968448112
            },
            {
              "sourceName": "GWSERVER02.corp.contoso.org",
              "objectName": "server11.corp.contoso.org",
              "objectType": "ESX",
              "version": "6.7.0",
              "buildNumber": "17167734",
              "solutionTags": [],
              "firstObserved": 1635968448112
            }
          ]
        }
      ],
      "totalRecords": 1,
      "timeTaken": 51
    }
  }
}

結果 CSV 文件：

"Finding Id","Issue Description","Risk if no Action Taken","Severity","Recommendations","Source Name","Object Name","Object Type","Host Version","Build","First Observed","Reference"
"risk#80703","PSOD with re-formatting a valid dedup metadata block.","Potential ESXi host crash","CRITICAL","This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804);This issue is resolved with VMware Cloud Foundation 4.1","GWSERVER01.corp.contoso.org","server01.corp.contoso.org","HostSystem","6.7.0","17499825","1629806351877","https://kb.vmware.com/s/article/80703"
"risk#80703","PSOD with re-formatting a valid dedup metadata block.","Potential ESXi host crash","CRITICAL","This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804);This issue is resolved with VMware Cloud Foundation 4.1","GWSERVER01.corp.contoso.org","server02.corp.contoso.org","HostSystem","6.7.0","17499825","1629806351877","https://kb.vmware.com/s/article/80703"
"risk#80703","PSOD with re-formatting a valid dedup metadata block.","Potential ESXi host crash","CRITICAL","This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804);This issue is resolved with VMware Cloud Foundation 4.1","GWSERVER02.corp.contoso.org","server10.corp.contoso.org","HostSystem","6.7.0","17167734","1635968448112","https://kb.vmware.com/s/article/80703"
"risk#80703","PSOD with re-formatting a valid dedup metadata block.","Potential ESXi host crash","CRITICAL","This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804);This issue is resolved with VMware Cloud Foundation 4.1","GWSERVER02.corp.contoso.org","server11.corp.contoso.org","HostSystem","6.7.0","17167734","1635968448112","https://kb.vmware.com/s/article/80703"

2 個解決方案

解決方案1
 0  2021-12-22 18:31:24

您可以從to_entries[]開始並從.value中提取並在最后一部分按@csv過濾，例如

jq -r '.data.activeFindings.findings | to_entries[] | .value | [ .findingId , .findingDescription, .findingImpact, .severity, (.recommendations | join(",")) , .sourceName, .objectName, .objectType, .version, .buildNumber, .firstObserved, (.kbLinkURLs | join(","))] | @csv'

演示

解決方案2
 0  2021-12-22 19:07:12

我會將 jq 與spyql結合起來，方法如下：

jq -c '.data.activeFindings.findings[]' full_sample.json | spyql "SELECT json->findingId AS 'Finding Id', json->findingDescription  AS 'Issue Description', json->findingImpact AS 'Risk if no Action Taken', json->severity AS Severity, ';'.join(json->recommendations) AS Recommendations, json->affectedObjects->sourceName AS 'Source Name', json->affectedObjects->objectName AS 'Object Name', json->affectedObjects->objectType AS 'Object Type', json->affectedObjects->version AS 'Host Version', json->affectedObjects->buildNumber AS Build, json->affectedObjects->firstObserved AS 'First Observed', ';'.join(json->kbLinkURLsVCF) AS Reference FROM json EXPLODE json->affectedObjects TO csv"      
Finding Id,Issue Description,Risk if no Action Taken,Severity,Recommendations,Source Name,Object Name,Object Type,Host Version,Build,First Observed,Reference
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server01.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server02.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server03.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server04.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server05.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server06.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server07.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER02.corp.contoso.org,server10.corp.contoso.org,ESX,6.7.0,17167734,1635968448112,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER02.corp.contoso.org,server11.corp.contoso.org,ESX,6.7.0,17167734,1635968448112,https://kb.vmware.com/s/article/80703

我正在使用 jq 提取我們需要的 JSON 部分，同時將 output 壓縮為 JSON 行（spyql 需要）。 Then, spyql takes care of the rest, namely joining arrays (expressions are python with some optional syntax sugar), renaming columns and generating the CSV.

免責聲明：我是spyql的作者

暫無
暫無

聲明:本站的技術帖子網頁，遵循CC BY-SA 4.0協議，如果您需要轉載，請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.

相關問題 用 jq 導出 JSON 到 CSV 如何使用 JQ 從 JSON 數據中導出 csv 通過 jq 導出 json 到 csv jQuery查詢格式的CSV導出 如何在jq中為CSV導出添加標題？ 如何使用 JQ 將數據導出到 csv 文件 如何從 json 導出 CSV 文件，並通過 JQ 進行乘法運算 使用 Pandas 包將 JSON 結果導出到 CSV 使用 jq 執行從 json 創建的導出命令 如何使用jq從json創建具有可變數組長度的csv
相關標簽
 
粵ICP備18138465號  © 2020-2024 STACKOOM.COM