从 jq 导出结果到 csv
[英]export results from jq to csv
问题描述
I'm having the following dataset which I'd like to export into a CSV:
我有以下数据集，我想将其导出到 CSV 中：
Dataset:数据集：
{
"data": {
"activeFindings": {
"findings": [
{
"findingId": "risk#80703",
"accountId": "00000000-000000-0000000-000000",
"products": [
"GWSERVER01"
],
"findingDisplayName": "risk#80703",
"severity": "CRITICAL",
"findingDescription": "PSOD with re-formatting a valid dedup metadata block.",
"findingImpact": "Potential ESXi host crash",
"recommendations": [
"This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523)",
"This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804)"
],
"kbLinkURLs": [
"https://kb.vmware.com/s/article/80703"
],
"recommendationsVCF": [
"This issue is resolved with VMware Cloud Foundation 4.1"
],
"kbLinkURLsVCF": [
"https://kb.vmware.com/s/article/80703"
],
"categoryName": "Storage",
"findingTypes": [
"UPGRADE"
],
"firstObserved": 1629806351877,
"totalAffectedObjectsCount": 12,
"affectedObjects": [
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server01.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server02.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server03.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server04.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server05.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server06.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server07.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
}
]
}
],
"totalRecords": 1,
"timeTaken": 56
}
}
}
{
"data": {
"activeFindings": {
"findings": [
{
"findingId": "risk#80703",
"accountId": "00000000-000000-0000000-000000",
"products": [
"GWSERVER02.corp.contoso.org"
],
"findingDisplayName": "risk#80703",
"severity": "CRITICAL",
"findingDescription": "PSOD with re-formatting a valid dedup metadata block.",
"findingImpact": "Potential ESXi host crash",
"recommendations": [
"This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523)",
"This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804)"
],
"kbLinkURLs": [
"https://kb.vmware.com/s/article/80703"
],
"recommendationsVCF": [
"This issue is resolved with VMware Cloud Foundation 4.1"
],
"kbLinkURLsVCF": [
"https://kb.vmware.com/s/article/80703"
],
"categoryName": "Storage",
"findingTypes": [
"UPGRADE"
],
"firstObserved": 1635968448112,
"totalAffectedObjectsCount": 2,
"affectedObjects": [
{
"sourceName": "GWSERVER02.corp.contoso.org",
"objectName": "server10.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17167734",
"solutionTags": [],
"firstObserved": 1635968448112
},
{
"sourceName": "GWSERVER02.corp.contoso.org",
"objectName": "server11.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17167734",
"solutionTags": [],
"firstObserved": 1635968448112
}
]
}
],
"totalRecords": 1,
"timeTaken": 51
}
}
}
And header would be as follow: header 如下：
"Finding Id","Issue Description","Risk if no Action Taken","Severity","Recommendations","Source Name","Object Name","Object Type","Host Version","Build","First Observed","Reference"
Header keys mapping as follow: Header 按键映射如下：
- Finding Id = findingId查找 ID = 查找 ID
- Issue Description = findingDescription问题描述 = 发现描述
- Risk if no Action Taken = findingImpact不采取行动的风险 = 发现影响
- Severity = severity严重性 = 严重性
- Recommendations = recommendations推荐=推荐
- Source Name = sourceName源名称 = 源名称
- Object Name = objectName Object 名称 = 对象名称
- Object Type = objectType Object 类型 = objectType
- Host Version = version主机版本 = 版本
- Build = buildNumber构建 = buildNumber
- First Observed = firstObserved第一个观察到 = 第一个观察到
- Reference = kbLinkURLs参考 = kbLinkURLs
Unfortunately, we have to perform an API call per each finding & product (eg: we're not able to pull all the findings for all products at once - the API does not allow us to perform such query and thus, we have to make several calls to get all the findings with its associated objects.)不幸的是，我们必须对每个发现和产品执行 API 调用（例如：我们无法一次提取所有产品的所有发现 - API 不允许我们执行此类查询，因此，我们必须进行多次调用以获取所有结果及其相关对象。）
With that said, what would be the preferred approach to export the data into a csv?话虽如此，将数据导出到 csv 的首选方法是什么？ Would using jq's @CSV work though we would have to loop through several nodes?尽管我们必须遍历多个节点，但使用 jq 的 @CSV 会起作用吗？
Any help/guidance would be appreciated.任何帮助/指导将不胜感激。
Thanks!谢谢！
Note 1: A stripped version of the dataset as requested by chepner注 1： chepner要求的数据集的剥离版本
{
"data": {
"activeFindings": {
"findings": [
{
"findingId": "risk#80703",
"accountId": "00000000-000000-0000000-000000",
"products": [
"GWSERVER01"
],
"findingDisplayName": "risk#80703",
"severity": "CRITICAL",
"findingDescription": "PSOD with re-formatting a valid dedup metadata block.",
"findingImpact": "Potential ESXi host crash",
"recommendations": [
"This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523)",
"This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804)"
],
"kbLinkURLs": [
"https://kb.vmware.com/s/article/80703"
],
"recommendationsVCF": [
"This issue is resolved with VMware Cloud Foundation 4.1"
],
"kbLinkURLsVCF": [
"https://kb.vmware.com/s/article/80703"
],
"categoryName": "Storage",
"findingTypes": [
"UPGRADE"
],
"firstObserved": 1629806351877,
"totalAffectedObjectsCount": 12,
"affectedObjects": [
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server01.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
{
"sourceName": "GWSERVER01.corp.contoso.org",
"objectName": "server02.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17499825",
"solutionTags": [],
"firstObserved": 1629806351877
},
]
}
],
"totalRecords": 1,
"timeTaken": 56
}
}
}
{
"data": {
"activeFindings": {
"findings": [
{
"findingId": "risk#80703",
"accountId": "00000000-000000-0000000-000000",
"products": [
"GWSERVER02.corp.contoso.org"
],
"findingDisplayName": "risk#80703",
"severity": "CRITICAL",
"findingDescription": "PSOD with re-formatting a valid dedup metadata block.",
"findingImpact": "Potential ESXi host crash",
"recommendations": [
"This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523)",
"This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804)"
],
"kbLinkURLs": [
"https://kb.vmware.com/s/article/80703"
],
"recommendationsVCF": [
"This issue is resolved with VMware Cloud Foundation 4.1"
],
"kbLinkURLsVCF": [
"https://kb.vmware.com/s/article/80703"
],
"categoryName": "Storage",
"findingTypes": [
"UPGRADE"
],
"firstObserved": 1635968448112,
"totalAffectedObjectsCount": 2,
"affectedObjects": [
{
"sourceName": "GWSERVER02.corp.contoso.org",
"objectName": "server10.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17167734",
"solutionTags": [],
"firstObserved": 1635968448112
},
{
"sourceName": "GWSERVER02.corp.contoso.org",
"objectName": "server11.corp.contoso.org",
"objectType": "ESX",
"version": "6.7.0",
"buildNumber": "17167734",
"solutionTags": [],
"firstObserved": 1635968448112
}
]
}
],
"totalRecords": 1,
"timeTaken": 51
}
}
}
And the resulted CSV file:结果 CSV 文件：
"Finding Id","Issue Description","Risk if no Action Taken","Severity","Recommendations","Source Name","Object Name","Object Type","Host Version","Build","First Observed","Reference"
"risk#80703","PSOD with re-formatting a valid dedup metadata block.","Potential ESXi host crash","CRITICAL","This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804);This issue is resolved with VMware Cloud Foundation 4.1","GWSERVER01.corp.contoso.org","server01.corp.contoso.org","HostSystem","6.7.0","17499825","1629806351877","https://kb.vmware.com/s/article/80703"
"risk#80703","PSOD with re-formatting a valid dedup metadata block.","Potential ESXi host crash","CRITICAL","This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804);This issue is resolved with VMware Cloud Foundation 4.1","GWSERVER01.corp.contoso.org","server02.corp.contoso.org","HostSystem","6.7.0","17499825","1629806351877","https://kb.vmware.com/s/article/80703"
"risk#80703","PSOD with re-formatting a valid dedup metadata block.","Potential ESXi host crash","CRITICAL","This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804);This issue is resolved with VMware Cloud Foundation 4.1","GWSERVER02.corp.contoso.org","server10.corp.contoso.org","HostSystem","6.7.0","17167734","1635968448112","https://kb.vmware.com/s/article/80703"
"risk#80703","PSOD with re-formatting a valid dedup metadata block.","Potential ESXi host crash","CRITICAL","This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804);This issue is resolved with VMware Cloud Foundation 4.1","GWSERVER02.corp.contoso.org","server11.corp.contoso.org","HostSystem","6.7.0","17167734","1635968448112","https://kb.vmware.com/s/article/80703"
2 个解决方案
解决方案1
0 2021-12-22 18:31:24
You can start with to_entries[] and extract from .value s and filter by @csv at the last part such as您可以从to_entries[]开始并从.value中提取并在最后一部分按@csv过滤，例如
jq -r '.data.activeFindings.findings | to_entries[] | .value | [ .findingId , .findingDescription, .findingImpact, .severity, (.recommendations | join(",")) , .sourceName, .objectName, .objectType, .version, .buildNumber, .firstObserved, (.kbLinkURLs | join(","))] | @csv'
解决方案2
0 2021-12-22 19:07:12
I would combine jq with spyql , here's how:我会将 jq 与spyql结合起来，方法如下：
jq -c '.data.activeFindings.findings[]' full_sample.json | spyql "SELECT json->findingId AS 'Finding Id', json->findingDescription AS 'Issue Description', json->findingImpact AS 'Risk if no Action Taken', json->severity AS Severity, ';'.join(json->recommendations) AS Recommendations, json->affectedObjects->sourceName AS 'Source Name', json->affectedObjects->objectName AS 'Object Name', json->affectedObjects->objectType AS 'Object Type', json->affectedObjects->version AS 'Host Version', json->affectedObjects->buildNumber AS Build, json->affectedObjects->firstObserved AS 'First Observed', ';'.join(json->kbLinkURLsVCF) AS Reference FROM json EXPLODE json->affectedObjects TO csv"
Finding Id,Issue Description,Risk if no Action Taken,Severity,Recommendations,Source Name,Object Name,Object Type,Host Version,Build,First Observed,Reference
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server01.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server02.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server03.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server04.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server05.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server06.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER01.corp.contoso.org,server07.corp.contoso.org,ESX,6.7.0,17499825,1629806351877,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER02.corp.contoso.org,server10.corp.contoso.org,ESX,6.7.0,17167734,1635968448112,https://kb.vmware.com/s/article/80703
risk#80703,PSOD with re-formatting a valid dedup metadata block.,Potential ESXi host crash,CRITICAL,This issue is resolved in VMware ESXi 6.7 upgrade to Patch 05 (17700523);This issue is resolved in VMware ESXi 7.0 upgrade to Update 1 (16850804),GWSERVER02.corp.contoso.org,server11.corp.contoso.org,ESX,6.7.0,17167734,1635968448112,https://kb.vmware.com/s/article/80703
I am using jq to extract the part of the JSON we need, while compressing the output to JSON lines (required by spyql).我正在使用 jq 提取我们需要的 JSON 部分，同时将 output 压缩为 JSON 行（spyql 需要）。 Then, spyql takes care of the rest, namely joining arrays (expressions are python with some optional syntax sugar), renaming columns and generating the CSV. Then, spyql takes care of the rest, namely joining arrays (expressions are python with some optional syntax sugar), renaming columns and generating the CSV.
Disclaimer: I am the author of spyql免责声明：我是spyql的作者
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.
相关问题
用 jq 导出 JSON 到 CSV - Export JSON to CSV with jq
如何使用 JQ 从 JSON 数据中导出 csv - How to export csv from JSON data with using JQ
通过 jq 导出 json 到 csv - Export json via jq to csv
jQuery查询格式的CSV导出 - jq query format csv export
如何在jq中为CSV导出添加标题？ - How to add a header to CSV export in jq?
如何使用 JQ 将数据导出到 csv 文件 - How to export data to csv file using JQ
如何从 json 导出 CSV 文件，并通过 JQ 进行乘法运算 - How can I export CSV file from json with the result of multiplication via JQ
使用 Pandas 包将 JSON 结果导出到 CSV - Export JSON results to CSV using Pandas package
使用 jq 执行从 json 创建的导出命令 - Execute an export command created from a json using jq
如何使用jq从json创建具有可变数组长度的csv - How to make a csv from json with variable array lengths using jq
相关标签