Parse AWS security group with jq and add new ingress rule

Question

The problem I'm trying to solve is to search for AWS SG group and add a new rule (eg: 10.10.0.0/16) to the same ingress rule block(port, protocal,cidr) where when the pattern is “CidrIp”: “10.219.0.0/16” matched.

Found match: - https://gist.github.com/mfang329/49575d6eb7ddb93e8f926648f9ba06e9

{
    "PrefixListIds": [],
    "FromPort": -1,
    "IpRanges": [{
            "CidrIp": "10.219.0.0/16"
        },
        {
            "Description": "testing for vpc transit connectivity",
            "CidrIp": "0.0.0.0/0"
        }
    ],
    "ToPort": -1,
    "IpProtocol": "icmp",
    "UserIdGroupPairs": [{
        "UserId": "abcde80269151",
        "Description": "default SG VPC - peering ",
        "GroupId": "sg-33511e41"
    }],
    "Ipv6Ranges": []
}

Change to - https://gist.github.com/mfang329/b5e892cf2fee2da4b7e67106cd15b3b2

{
    "PrefixListIds": [],
    "FromPort": -1,
    "IpRanges": [{
            "CidrIp": "10.219.0.0/16"
        },
        {
            "CidrIp": "10.10.0.0/16"
        },
        {
            "Description": "testing for vpc transit connectivity",
            "CidrIp": "0.0.0.0/0"
        }
    ],
    "ToPort": -1,
    "IpProtocol": "icmp",
    "UserIdGroupPairs": [{
        "UserId": "abcde80269151",
        "Description": "default SG VPC - peering ",
        "GroupId": "sg-33511e41"
    }],
    "Ipv6Ranges": []
}

Modify the SG with the following, but how do I express the jq to query these information and use them as the input for following aws cli? I know there are commands flags with JQ but I would like to what is the simplest solution to attack this problem?

aws ec2 revoke-security-group-ingress \
 — group-id sg-33511e41 \
 — port -1 \
 — protocol icmp \
 — cidr 10.10.0.0/16;

---

Full security group JSON format - https://gist.github.com/mfang329/a1871731fe82e5255ccc571648ad4886 *

Answer 1

If your intention is to create a new CidrIp rule added when the value is 10.219.0.0/16 , do the following. Note that this does not preserve the order of the elements in the array and adds the new IP to the tail end

jq 'if   .IpRanges[] | select(.CidrIp | contains("10.219.0.0/16") )
    then .IpRanges += [ { "cidrIp" : "10.10.0.0/16" } ]
    else . end' ingressJSON

To store this output in a variable and to be later used in your aws command, store the output above to a shell variable. And pass it to the command with proper quotes as "$awsRule"

awsRule=$(jq 'if   .IpRanges[] | select(.CidrIp | contains("10.219.0.0/16") )
              then .IpRanges += [ { "cidrIp" : "10.10.0.0/16" } ]
              else . end' ingressJSON)

Answer 2

The answer I eventually figured out was to construct a simple SG json format that I use to feed it back to the aws ec2 statement. This approach took me awhile to figure out but it was quite elegant and I think it was what I was going for.

# search for the matching pattern and write to the output file
aws ec2 describe-security-groups --region ${i} |   \
  jq -r --arg e_cidr "${existing_cidr}" \
     --arg n_cidr "${new_cidr}" \
     '.SecurityGroups[] | .GroupId as $gid | .IpPermissions[] | .FromPort as $port | .IpProtocol as $protocol | 
     .IpRanges[] | select(.CidrIp == $e_cidr) | .Description as $desc |
     [{ GroupId:$gid, IpProtocol:$protocol, FromPort:$port, ToPort:$port, IpRanges:[{CidrIp:$n_cidr, Description:$desc}] }] ' -c   \
     | tee -a ${regional_file} 

This will produce a json file similar to this format,

[{
"GroupId": "sg-60d78e12",
"IpProtocol": "tcp",
"FromPort": 443,
"ToPort": 443,
"IpRanges": [
  {
    "CidrIp": "12.179.53.18/32",
    "Description": "Chicago Primary"
  }
]
}]

Then what I did was to drop the "GroupId" key value from the json statement, then feed into the aws ec2 command line to add the new ingress rule with the same protocol and port as of the original cidr block. This work nicely and took a little while to get to this conclusion.

        while read -r sg_pattern
        do
        gid=$( echo $sg_pattern | jq '.[]|.GroupId' -r )

        # remove the GroupId key to prepare the ip_permissions json
        ip_permissions=$( echo $sg_pattern | jq '.[] | [ del(.GroupId) ]' -r )
        echo " ---> adding new SG rule for ${ip_permissions}." 
        aws ec2 authorize-security-group-ingress --group-id $gid --ip-permissions "${ip_permissions}"

    done < ${regional_file}