[英]How to improve performance on a slow jq script?
I have a JSON doc that looks like: 我有一个JSON文档,看起来像:
{
"SecurityGroups": [
{
"GroupName": "database",
"GroupId": "sg-xxxxxx",
"VpcId": "vpc-yyyyyyy",
"IpPermissions": [
{
"FromPort": 22,
"ToPort": 22,
"IpProtocol": "tcp",
"IpRanges": [
{
"CidrIp": "10.200.0.0/16"
},
{
"CidrIp": "10.200.30.79/32"
},
{
"CidrIp": "10.200.42.0/24"
}
],
"UserIdGroupPairs": []
},
{
"FromPort": 5555,
"ToPort": 5555,
"IpProtocol": "tcp",
"IpRanges": [
{
"CidrIp": "10.200.0.0/16"
},
{
"CidrIp": "10.200.0.155/32"
}
],
"UserIdGroupPairs": []
},
{
"FromPort": 4506,
"ToPort": 4506,
"IpProtocol": "tcp",
"IpRanges": [
{
"CidrIp": "10.200.0.0/16"
}
],
"UserIdGroupPairs": []
}
]
}
]
}
The output I need to generate is as follow: 我需要生成的输出如下:
sg-xxxxxx|database|22|22|tcp|10.200.0.0/16
sg-xxxxxx|database|22|22|tcp|10.200.30.79/32
sg-xxxxxx|database|22|22|tcp|10.200.42.0/24
sg-xxxxxx|database|5555|5555|tcp|10.200.0.0/16
sg-xxxxxx|database|5555|5555|tcp|10.200.0.155/32
sg-xxxxxx|database|4506|4506|tcp|10.200.0.0/16
I'm able to achieve that by using using jq first to generate a list of GroupId's and then loop through the list to filter data into jq twice. 我可以通过先使用jq生成GroupId的列表,然后遍历该列表两次将数据过滤到jq中来实现此目的。 Here's how I did it: 这是我的操作方式:
cat json.in | jq -r '.SecurityGroups[]|"\(.GroupId) \(.GroupName)"' | while read groupid groupname
do
cat json.in | jq ".SecurityGroups[]|{GroupId,IpPermissions,IpPermissionsEgress}|select(.GroupId == \"$groupid\")" | jq -r '.IpPermissions[]|"\(.FromPort)|\(.ToPort)|\(.IpProtocol)|\(.IpRanges[].CidrIp)"' | sed "s/^/$groupid|$groupname|/"
done
My solution is slow and I would like to improve on it, Any pointers? 我的解决方案很慢,我想对其进行改进,有什么建议吗?
Here is a more efficient approach. 这是一种更有效的方法。 With the -r
option, the following filter 使用-r
选项,以下过滤器
.SecurityGroups[]
| .GroupId as $gid
| .GroupName as $gname
| (.IpPermissions[], .IpPermissionsEgress[]?)
| .FromPort as $from
| .ToPort as $to
| .IpProtocol as $pro
| .IpRanges[]
| "\($gid)|\($gname)|\($from)|\($to)|\($pro)|\(.CidrIp)"
with the sample data produces 与样本数据产生
sg-xxxxxx|database|22|22|tcp|10.200.0.0/16
sg-xxxxxx|database|22|22|tcp|10.200.30.79/32
sg-xxxxxx|database|22|22|tcp|10.200.42.0/24
sg-xxxxxx|database|5555|5555|tcp|10.200.0.0/16
sg-xxxxxx|database|5555|5555|tcp|10.200.0.155/32
sg-xxxxxx|database|4506|4506|tcp|10.200.0.0/16
Note that this includes .IpPermissionsEgress[]?
请注意,这包括.IpPermissionsEgress[]?
because although it's absent from your sample data and unused in the second part of your script it is nevertheless present in the first part of your sample script so I think you may have intended to include it. 因为尽管示例数据中缺少这些数据,但在脚本的第二部分中未使用它,但是尽管如此,它仍存在于示例脚本的第一部分中,所以我认为您可能打算将其包括在内。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.