使用JQ解析AWS CLI输出安全组
[英]Parse aws cli output security groups with JQ
问题描述
IM gettin' crazy using "JQ" for parse a json output from the AWS CLI. 
IM使用“ JQ”解析来自AWS CLI的json输出，从而发疯。
Maybe i have not yet understand how to use JQ properly, but id like to parse 2 (or more) security groups having one string formatted for every rule, both inbound and outbound 也许我还不太了解如何正确使用JQ，但是id想要解析2个（或更多）安全组，每个安全组的入站和出站规则都使用一个字符串格式
The command AWS CLI is about this: 命令AWS CLI与此有关：
aws ec2 describe-security-groups --group-ids sg-0000001 sg-0000002
And the output json is about this (i have random edited some infos for privacy): 输出的json就是关于这个的（我已经随机编辑了一些隐私信息）：
{
"SecurityGroups": [
{
"IpPermissionsEgress": [
{
"IpProtocol": "-1",
"IpRanges": [
{
"CidrIp": "0.0.0.0/0"
}
],
"UserIdGroupPairs": []
}
],
"Description": "server-db",
"Tags": [
{
"Value": "server-db",
"Key": "Client"
},
{
"Value": "server-db",
"Key": "Name"
}
],
"IpPermissions": [
{
"ToPort": 3389,
"IpProtocol": "tcp",
"IpRanges": [
{
"CidrIp": "10.12.0.0/16"
},
{
"CidrIp": "192.168.10.10/32"
}
],
"UserIdGroupPairs": [],
"FromPort": 3389
},
{
"ToPort": 5666,
"IpProtocol": "tcp",
"IpRanges": [
{
"CidrIp": "192.168.10.10/32"
}
],
"UserIdGroupPairs": [],
"FromPort": 5666
},
{
"IpProtocol": "-1",
"IpRanges": [],
"UserIdGroupPairs": [
{
"UserId": "121211212121",
"GroupId": "sg-00000001"
}
]
},
{
"IpProtocol": "-1",
"IpRanges": [],
"UserIdGroupPairs": [
{
"UserId": "121211212121",
"GroupId": "sg-000000001"
}
]
},
{
"ToPort": -1,
"IpProtocol": "icmp",
"IpRanges": [
{
"CidrIp": "10.12.0.0/16"
},
{
"CidrIp": "192.168.10.10/32"
}
],
"UserIdGroupPairs": [],
"FromPort": -1
}
],
"GroupName": "server-db",
"VpcId": "vpc-0000001",
"OwnerId": "121211212121",
"GroupId": "sg-000000001"
},
{
"IpPermissionsEgress": [
{
"IpProtocol": "-1",
"IpRanges": [
{
"CidrIp": "0.0.0.0/0"
}
],
"UserIdGroupPairs": []
}
],
"Description": "server-as",
"Tags": [
{
"Value": "server-as",
"Key": "Name"
},
{
"Value": "server",
"Key": "Client"
}
],
"IpPermissions": [
{
"IpProtocol": "-1",
"IpRanges": [],
"UserIdGroupPairs": [
{
"UserId": "121211212121",
"GroupId": "sg-00000001"
}
]
},
{
"ToPort": 22,
"IpProtocol": "tcp",
"IpRanges": [
{
"CidrIp": "10.12.0.0/16"
},
{
"CidrIp": "192.168.10.10/32"
}
],
"UserIdGroupPairs": [],
"FromPort": 22
},
{
"ToPort": 443,
"IpProtocol": "tcp",
"IpRanges": [
{
"CidrIp": "10.12.0.0/16"
},
{
"CidrIp": "192.168.60.10/32"
},
{
"CidrIp": "192.168.160.10/32"
},
{
"CidrIp": "192.168.130.10/32"
},
{
"CidrIp": "192.168.130.50/32"
},
{
"CidrIp": "192.168.130.150/32"
},
{
"CidrIp": "192.168.10.10/32"
},
{
"CidrIp": "192.168.80.150/32"
},
{
"CidrIp": "192.168.80.152/32"
},
{
"CidrIp": "192.168.80.155/32"
},
{
"CidrIp": "192.168.80.158/32"
}
],
"UserIdGroupPairs": [],
"FromPort": 443
},
{
"IpProtocol": "-1",
"IpRanges": [],
"UserIdGroupPairs": [
{
"UserId": "121211212121",
"GroupId": "sg-00000002"
}
]
},
{
"ToPort": -1,
"IpProtocol": "icmp",
"IpRanges": [
{
"CidrIp": "10.12.0.0/16"
},
{
"CidrIp": "192.168.10.10/32"
}
],
"UserIdGroupPairs": [],
"FromPort": -1
}
],
"GroupName": "server-as",
"VpcId": "vpc-00000001",
"OwnerId": "121211212121",
"GroupId": "sg-00000001"
}
]
}
The format that id like to have could be something as: id喜欢的格式可以是：
SecurityGroupId - GroupName - InBound/OutBound - IpProtocol - Port - SourceRanges/DestinationRanges
AnyOne could help me? 有人可以帮助我吗？ Thanks 谢谢
2 个解决方案
解决方案1
0 2014-10-30 14:53:58
Resolved with PERL: 使用PERL解决：
use JSON qw( decode_json );
sub creatab{
my $json = $dump; #dump from the AWS CLI
my $decoded = decode_json($json);
my @secgrp = @{ $decoded->{'SecurityGroups'} };
foreach my $f ( @secgrp ) {
$description=$f->{"Description"};
$groupname=$f->{"GroupName"};
$vpcid=$f->{"VpcId"}; if ($vpcid eq "") {$vpcid = "EC2"}
$groupid=$f->{"GroupId"};
#------INBOUND RULES------------------------
my @ipperm = @{ $f->{'IpPermissions'} };
foreach my $g ( @ipperm ) {
$toport=$g->{'ToPort'};
$fromport=$g->{'FromPort'};
$proto=$g->{'IpProtocol'};
my @cidr = @{ $g->{'IpRanges'} };
foreach my $h ( @cidr ) {
$cidr=$h->{'CidrIp'};
if ($proto==-1) {$fromport="ALLPORTS"; $toport="ALLPORTS"; $proto="ALLPROTO";}
$tabella .= "$groupid|$groupname|$description|$vpcid|INBOUND|$cidr|$fromport|$toport|$proto\n";
}
my @useridgrouppairs = @{ $g->{'UserIdGroupPairs'} };
foreach my $h ( @useridgrouppairs ) {
$useridgrouppairs=$h->{'GroupId'};
if ($proto==-1) {$fromport="ALLPORTS"; $toport="ALLPORTS"; $proto="ALLPROTO";}
$tabella .= "$groupid|$groupname|$description|$vpcid|INBOUND|$useridgrouppairs|$fromport|$toport|$proto\n";
}
}
#-------------------------------------------
#-------------OUTBOUND RULES----------------
my @ipperm = @{ $f->{'IpPermissionsEgress'} };
foreach my $g ( @ipperm ) {
$toport=$g->{'ToPort'};
$fromport=$g->{'FromPort'};
$proto=$g->{'IpProtocol'};
my @cidr = @{ $g->{'IpRanges'} };
foreach my $h ( @cidr ) {
$cidr=$h->{'CidrIp'};
if ($proto==-1) {$fromport="ALLPORTS"; $toport="ALLPORTS"; $proto="ALLPROTO";}
$tabella .= "$groupid|$groupname|$description|$vpcid|OUTBOUND|$cidr|$fromport|$toport|$proto\n";
}
my @useridgrouppairs = @{ $g->{'UserIdGroupPairs'} };
foreach my $h ( @useridgrouppairs ) {
$useridgrouppairs=$h->{'GroupId'};
if ($proto==-1) {$fromport="ALLPORTS"; $toport="ALLPORTS"; $proto="ALLPROTO";}
$tabella .= "$groupid|$groupname|$description|$vpcid|OUTBOUND|$useridgrouppairs|$fromport|$toport|$proto\n";
}
}
#------------------------------------------
}
return $tabella;
}
解决方案2
0 2017-08-16 03:25:27
Here is a jq solution 这是一个jQuery解决方案
def ports:
if .IpProtocol == "-1"
then {FromPort:"ALLPORTS", ToPort:"ALLPORTS", IpProtocol:"ALLPROTO"}
else {FromPort, ToPort, IpProtocol}
end
;
def tabella($g; $p; $dir; $ip):
[ $g.GroupId, $g.GroupName, $g.Description, $g.VpcId, $dir, $ip,
$p.FromPort, $p.ToPort, $p.IpProtocol ]
;
def creatab:
.SecurityGroups[]
| { GroupId, GroupName, Description, VpcId } as $g
| (
.IpPermissions[]
| ports as $p
| ( .IpRanges[] | tabella($g; $p; "INBOUND"; .CidrIp) ),
( .UserIdGroupPairs[] | tabella($g; $p; "INBOUND"; .GroupId) )
),
(
.IpPermissionsEgress[]
| ports as $p
| ( .IpRanges[] | tabella($g; $p; "OUTBOUND"; .CidrIp) ),
( .UserIdGroupPairs[] | tabella($g; $p; "OUTBOUND"; .GroupId) )
)
;
creatab
| map(tostring)
| join("|")
when run as 当以
jq -M -r -f filter.jq data.json
it will produce the same output as Marco's perl solution 它将产生与Marco的perl解决方案相同的输出
sg-000000001|server-db|server-db|vpc-0000001|INBOUND|10.12.0.0/16|3389|3389|tcp
sg-000000001|server-db|server-db|vpc-0000001|INBOUND|192.168.10.10/32|3389|3389|tcp
sg-000000001|server-db|server-db|vpc-0000001|INBOUND|192.168.10.10/32|5666|5666|tcp
sg-000000001|server-db|server-db|vpc-0000001|INBOUND|sg-00000001|ALLPORTS|ALLPORTS|ALLPROTO
sg-000000001|server-db|server-db|vpc-0000001|INBOUND|sg-000000001|ALLPORTS|ALLPORTS|ALLPROTO
sg-000000001|server-db|server-db|vpc-0000001|INBOUND|10.12.0.0/16|-1|-1|icmp
sg-000000001|server-db|server-db|vpc-0000001|INBOUND|192.168.10.10/32|-1|-1|icmp
sg-000000001|server-db|server-db|vpc-0000001|OUTBOUND|0.0.0.0/0|ALLPORTS|ALLPORTS|ALLPROTO
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|sg-00000001|ALLPORTS|ALLPORTS|ALLPROTO
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|10.12.0.0/16|22|22|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.10.10/32|22|22|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|10.12.0.0/16|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.60.10/32|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.160.10/32|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.130.10/32|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.130.50/32|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.130.150/32|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.10.10/32|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.80.150/32|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.80.152/32|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.80.155/32|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.80.158/32|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|sg-00000002|ALLPORTS|ALLPORTS|ALLPROTO
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|10.12.0.0/16|-1|-1|icmp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.10.10/32|-1|-1|icmp
sg-00000001|server-as|server-as|vpc-00000001|OUTBOUND|0.0.0.0/0|ALLPORTS|ALLPORTS|ALLPROTO
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.
相关问题
使用jq解析AWS CLI的json输出 - Using jq to parse json output of AWS CLI
使用jq解析AWS CLI工具的json输出 - Using jq to parse json output of AWS CLI tools
如何使用jq解析AWS CLI json输出以获取卷ID数组？ - How to parse AWS CLI json output using jq to get array of volume-ids?
使用jq解析AWS安全组并添加新的入口规则 - Parse AWS security group with jq and add new ingress rule
用jq解析jolokia输出 - Parse jolokia output with jq
使用jq解析json输出 - Parse json output using jq
通过 jq 将 aws cli 解析为 CSV - Parsing aws cli via jq into CSV
如何使用 jq 解析包含这个布尔值的 JSON 输出？ - How to parse this boolean contained JSON output with jq?
用jq解析json的特定输出 - Parse specific output from json with jq
jq 解析 json 并生成 json output - jq to parse json and generate json output
相关标签