堆棧內存溢出
  簡體   English   中英

使用JQ解析AWS CLI輸出安全組

[英]Parse aws cli output security groups with JQ

 原文  2014-10-24 07:27:00  9   2    json/ amazon-web-services/ jq

問題描述

IM使用“ JQ”解析來自AWS CLI的json輸出，從而發瘋。

也許我還不太了解如何正確使用JQ，但是id想要解析2個（或更多）安全組，每個安全組的入站和出站規則都使用一個字符串格式

命令AWS CLI與此有關： 

aws ec2 describe-security-groups --group-ids sg-0000001 sg-0000002

輸出的json就是關於這個的（我已經隨機編輯了一些隱私信息）： 

{
    "SecurityGroups": [
        {
            "IpPermissionsEgress": [
                {
                    "IpProtocol": "-1",
                    "IpRanges": [
                        {
                            "CidrIp": "0.0.0.0/0"
                        }
                    ],
                    "UserIdGroupPairs": []
                }
            ],
            "Description": "server-db",
            "Tags": [
                {
                    "Value": "server-db",
                    "Key": "Client"
                },
                {
                    "Value": "server-db",
                    "Key": "Name"
                }
            ],
            "IpPermissions": [
                {
                    "ToPort": 3389,
                    "IpProtocol": "tcp",
                    "IpRanges": [
                        {
                            "CidrIp": "10.12.0.0/16"
                        },
                        {
                            "CidrIp": "192.168.10.10/32"
                        }
                    ],
                    "UserIdGroupPairs": [],
                    "FromPort": 3389
                },
                {
                    "ToPort": 5666,
                    "IpProtocol": "tcp",
                    "IpRanges": [
                        {
                            "CidrIp": "192.168.10.10/32"
                        }
                    ],
                    "UserIdGroupPairs": [],
                    "FromPort": 5666
                },
                {
                    "IpProtocol": "-1",
                    "IpRanges": [],
                    "UserIdGroupPairs": [
                        {
                            "UserId": "121211212121",
                            "GroupId": "sg-00000001"
                        }
                    ]
                },
                {
                    "IpProtocol": "-1",
                    "IpRanges": [],
                    "UserIdGroupPairs": [
                        {
                            "UserId": "121211212121",
                            "GroupId": "sg-000000001"
                        }
                    ]
                },
                {
                    "ToPort": -1,
                    "IpProtocol": "icmp",
                    "IpRanges": [
                        {
                            "CidrIp": "10.12.0.0/16"
                        },
                        {
                            "CidrIp": "192.168.10.10/32"
                        }
                    ],
                    "UserIdGroupPairs": [],
                    "FromPort": -1
                }
            ],
            "GroupName": "server-db",
            "VpcId": "vpc-0000001",
            "OwnerId": "121211212121",
            "GroupId": "sg-000000001"
        },
        {
            "IpPermissionsEgress": [
                {
                    "IpProtocol": "-1",
                    "IpRanges": [
                        {
                            "CidrIp": "0.0.0.0/0"
                        }
                    ],
                    "UserIdGroupPairs": []
                }
            ],
            "Description": "server-as",
            "Tags": [
                {
                    "Value": "server-as",
                    "Key": "Name"
                },
                {
                    "Value": "server",
                    "Key": "Client"
                }
            ],
            "IpPermissions": [
                {
                    "IpProtocol": "-1",
                    "IpRanges": [],
                    "UserIdGroupPairs": [
                        {
                            "UserId": "121211212121",
                            "GroupId": "sg-00000001"
                        }
                    ]
                },
                {
                    "ToPort": 22,
                    "IpProtocol": "tcp",
                    "IpRanges": [
                        {
                            "CidrIp": "10.12.0.0/16"
                        },
                        {
                            "CidrIp": "192.168.10.10/32"
                        }
                    ],
                    "UserIdGroupPairs": [],
                    "FromPort": 22
                },
                {
                    "ToPort": 443,
                    "IpProtocol": "tcp",
                    "IpRanges": [
                        {
                            "CidrIp": "10.12.0.0/16"
                        },
                        {
                            "CidrIp": "192.168.60.10/32"
                        },
                        {
                            "CidrIp": "192.168.160.10/32"
                        },
                        {
                            "CidrIp": "192.168.130.10/32"
                        },
                        {
                            "CidrIp": "192.168.130.50/32"
                        },
                        {
                            "CidrIp": "192.168.130.150/32"
                        },
                        {
                            "CidrIp": "192.168.10.10/32"
                        },
                        {
                            "CidrIp": "192.168.80.150/32"
                        },
                        {
                            "CidrIp": "192.168.80.152/32"
                        },
                        {
                            "CidrIp": "192.168.80.155/32"
                        },
                        {
                            "CidrIp": "192.168.80.158/32"
                        }
                    ],
                    "UserIdGroupPairs": [],
                    "FromPort": 443
                },
                {
                    "IpProtocol": "-1",
                    "IpRanges": [],
                    "UserIdGroupPairs": [
                        {
                            "UserId": "121211212121",
                            "GroupId": "sg-00000002"
                        }
                    ]
                },
                {
                    "ToPort": -1,
                    "IpProtocol": "icmp",
                    "IpRanges": [
                        {
                            "CidrIp": "10.12.0.0/16"
                        },
                        {
                            "CidrIp": "192.168.10.10/32"
                        }
                    ],
                    "UserIdGroupPairs": [],
                    "FromPort": -1
                }
            ],
            "GroupName": "server-as",
            "VpcId": "vpc-00000001",
            "OwnerId": "121211212121",
            "GroupId": "sg-00000001"
        }
    ]
}

id喜歡的格式可以是： 

SecurityGroupId - GroupName - InBound/OutBound - IpProtocol - Port - SourceRanges/DestinationRanges

有人可以幫助我嗎？ 謝謝

2 個解決方案

解決方案1
 0  2014-10-30 14:53:58

使用PERL解決： 

use JSON qw( decode_json );

sub creatab{
my $json = $dump; #dump from the AWS CLI
my $decoded = decode_json($json);

my @secgrp = @{ $decoded->{'SecurityGroups'} };
foreach my $f ( @secgrp ) {
 $description=$f->{"Description"};
 $groupname=$f->{"GroupName"};
 $vpcid=$f->{"VpcId"}; if ($vpcid eq "") {$vpcid = "EC2"}
 $groupid=$f->{"GroupId"};

#------INBOUND RULES------------------------
 my @ipperm = @{ $f->{'IpPermissions'} };
 foreach my $g ( @ipperm ) {
  $toport=$g->{'ToPort'};
  $fromport=$g->{'FromPort'};
  $proto=$g->{'IpProtocol'};

  my @cidr = @{ $g->{'IpRanges'} };
  foreach my $h ( @cidr ) {
   $cidr=$h->{'CidrIp'};
    if ($proto==-1) {$fromport="ALLPORTS"; $toport="ALLPORTS"; $proto="ALLPROTO";}
    $tabella .= "$groupid|$groupname|$description|$vpcid|INBOUND|$cidr|$fromport|$toport|$proto\n";
  }

  my @useridgrouppairs = @{ $g->{'UserIdGroupPairs'} };
  foreach my $h ( @useridgrouppairs ) {
   $useridgrouppairs=$h->{'GroupId'};
    if ($proto==-1) {$fromport="ALLPORTS"; $toport="ALLPORTS"; $proto="ALLPROTO";}
    $tabella .= "$groupid|$groupname|$description|$vpcid|INBOUND|$useridgrouppairs|$fromport|$toport|$proto\n";
  }
 }
#-------------------------------------------
#-------------OUTBOUND RULES----------------
 my @ipperm = @{ $f->{'IpPermissionsEgress'} };
 foreach my $g ( @ipperm ) {
  $toport=$g->{'ToPort'};
  $fromport=$g->{'FromPort'};
  $proto=$g->{'IpProtocol'};

  my @cidr = @{ $g->{'IpRanges'} };
  foreach my $h ( @cidr ) {
   $cidr=$h->{'CidrIp'};
    if ($proto==-1) {$fromport="ALLPORTS"; $toport="ALLPORTS"; $proto="ALLPROTO";}
    $tabella .= "$groupid|$groupname|$description|$vpcid|OUTBOUND|$cidr|$fromport|$toport|$proto\n";
  }

  my @useridgrouppairs = @{ $g->{'UserIdGroupPairs'} };
  foreach my $h ( @useridgrouppairs ) {
   $useridgrouppairs=$h->{'GroupId'};
    if ($proto==-1) {$fromport="ALLPORTS"; $toport="ALLPORTS"; $proto="ALLPROTO";}
    $tabella .= "$groupid|$groupname|$description|$vpcid|OUTBOUND|$useridgrouppairs|$fromport|$toport|$proto\n";
  }
 }
#------------------------------------------
}
return $tabella;
}

解決方案2
 0  2017-08-16 03:25:27

這是一個jQuery解決方案 

def ports:
   if .IpProtocol == "-1"
   then {FromPort:"ALLPORTS", ToPort:"ALLPORTS", IpProtocol:"ALLPROTO"}
   else {FromPort, ToPort, IpProtocol}
   end
;

def tabella($g; $p; $dir; $ip):
    [ $g.GroupId, $g.GroupName, $g.Description, $g.VpcId, $dir, $ip,
      $p.FromPort, $p.ToPort, $p.IpProtocol ]
;

def creatab:
      .SecurityGroups[]
    | { GroupId, GroupName, Description, VpcId } as $g
    | (
          .IpPermissions[]
        | ports as $p
        | ( .IpRanges[]         | tabella($g; $p; "INBOUND"; .CidrIp) ),
          ( .UserIdGroupPairs[] | tabella($g; $p; "INBOUND"; .GroupId) )
      ),
      (
          .IpPermissionsEgress[]
        | ports as $p
        | ( .IpRanges[]         | tabella($g; $p; "OUTBOUND"; .CidrIp) ),
          ( .UserIdGroupPairs[] | tabella($g; $p; "OUTBOUND"; .GroupId) )
      )
;

  creatab
| map(tostring)
| join("|")

當以 

jq -M -r -f filter.jq data.json

它將產生與Marco的perl解決方案相同的輸出 

sg-000000001|server-db|server-db|vpc-0000001|INBOUND|10.12.0.0/16|3389|3389|tcp
sg-000000001|server-db|server-db|vpc-0000001|INBOUND|192.168.10.10/32|3389|3389|tcp
sg-000000001|server-db|server-db|vpc-0000001|INBOUND|192.168.10.10/32|5666|5666|tcp
sg-000000001|server-db|server-db|vpc-0000001|INBOUND|sg-00000001|ALLPORTS|ALLPORTS|ALLPROTO
sg-000000001|server-db|server-db|vpc-0000001|INBOUND|sg-000000001|ALLPORTS|ALLPORTS|ALLPROTO
sg-000000001|server-db|server-db|vpc-0000001|INBOUND|10.12.0.0/16|-1|-1|icmp
sg-000000001|server-db|server-db|vpc-0000001|INBOUND|192.168.10.10/32|-1|-1|icmp
sg-000000001|server-db|server-db|vpc-0000001|OUTBOUND|0.0.0.0/0|ALLPORTS|ALLPORTS|ALLPROTO
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|sg-00000001|ALLPORTS|ALLPORTS|ALLPROTO
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|10.12.0.0/16|22|22|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.10.10/32|22|22|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|10.12.0.0/16|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.60.10/32|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.160.10/32|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.130.10/32|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.130.50/32|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.130.150/32|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.10.10/32|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.80.150/32|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.80.152/32|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.80.155/32|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.80.158/32|443|443|tcp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|sg-00000002|ALLPORTS|ALLPORTS|ALLPROTO
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|10.12.0.0/16|-1|-1|icmp
sg-00000001|server-as|server-as|vpc-00000001|INBOUND|192.168.10.10/32|-1|-1|icmp
sg-00000001|server-as|server-as|vpc-00000001|OUTBOUND|0.0.0.0/0|ALLPORTS|ALLPORTS|ALLPROTO

暫無
暫無

聲明:本站的技術帖子網頁，遵循CC BY-SA 4.0協議，如果您需要轉載，請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.

相關問題 使用jq解析AWS CLI的json輸出 使用jq解析AWS CLI工具的json輸出 如何使用jq解析AWS CLI json輸出以獲取卷ID數組？ 使用jq解析AWS安全組並添加新的入口規則 用jq解析jolokia輸出 使用jq解析json輸出 通過 jq 將 aws cli 解析為 CSV 如何使用 jq 解析包含這個布爾值的 JSON 輸出？ 用jq解析json的特定輸出 jq 解析 json 並生成 json output
相關標簽
 
粵ICP備18138465號  © 2020-2024 STACKOOM.COM