使用 JQ 过滤 cloudformation 堆栈资源

Question

I'm trying to write a JQ-filter for filtering specific resources from an AWS cloudformation template based on resource properties.

For example, when starting from the following (shortened) cloudformation template:

{
"Resources": {
"vpc001": {
  "Type": "AWS::EC2::VPC",
  "Properties": {
    "CidrBlock": "10.1.0.0/16",
    "InstanceTenancy": "default",
    "EnableDnsSupport": "true",
    "EnableDnsHostnames": "true"
  }
},
"ig001": {
  "Type": "AWS::EC2::InternetGateway",
  "Properties": {
    "Tags": [
      {
        "Key": "Name",
        "Value": "ig001"
      }
    ]
  }
}
}
}

I would like to construct a jq-filter enabling me to filter out specific resources based on (one or multiple) of their property fields.

For example:

when filtering for Type="AWS::EC2::Inte.netGateway" the result should be

{
 "Resources": {
"ig001": {
  "Type": "AWS::EC2::InternetGateway",
  "Properties": {
    "Tags": [
      {
        "Key": "Name",
        "Value": "ig001"
      }
    ]
  }
}
}
}

An added bonus would be to be able to filter on a 'OR'-ed combination of values. As such a filter for "AWS::EC2::Inte.netGateway" OR "AWS::EC2::VPC" should yield the original document.

Any suggestion or insight would be greatly appreciated.

Tx!

Answer 1

@hek2mgl's suggestion may be sufficient for your purposes, but it doesn't quite produce the answer you requested. Here's one very similar solution that does. It uses a generalization of jq's map() and map_values() filters that is often useful anyway:

def mapper(f):
  if type == "array" then map(f)
  elif type == "object" then
  . as $in
  | reduce keys[] as $key
      ({};
       [$in[$key] | f ] as $value
       | if $value | length == 0 then . else . + {($key): $value[0]}
         end)
  else .
  end;

.Resources |= mapper(select(.Type=="AWS::EC2::VPC"))

Using your example input:

$ jq -f resources.jq resources.json
{
  "Resources": {
    "vpc001": {
      "Type": "AWS::EC2::VPC",
      "Properties": {
        "CidrBlock": "10.1.0.0/16",
        "InstanceTenancy": "default",
        "EnableDnsSupport": "true",
        "EnableDnsHostnames": "true"
      }
    }
  }

As @hek2mgl pointed out, it's now trivial to specify a more complex selection criterion. }

Answer 2

Use the select() function:

jq '.Resources[]|select(.Type=="AWS::EC2::VPC")' aws.json

You can use or if you want to filter by multiple conditions, like this:

jq '.Resources[]|select(.Type=="AWS::EC2::VPC" or .Type=="foo")' aws.json

Answer 3

Use aws cli's --query parameter. Completely eliminates the need for jq. http://docs.aws.amazon.com/cli/latest/userguide/controlling-output.html#controlling-output-filter

Answer 4

Here is a solution which uses a separate function to select all resources matching a specified condition which is passed a {key,value} pair for each resource.

def condition:
  .value.Type == "AWS::EC2::VPC"
;

{
  Resources: .Resources | with_entries(select(condition))
}

Output from sample data:

{
  "Resources": {
    "vpc001": {
      "Type": "AWS::EC2::VPC",
      "Properties": {
        "CidrBlock": "10.1.0.0/16",
        "InstanceTenancy": "default",
        "EnableDnsSupport": "true",
        "EnableDnsHostnames": "true"
      }
    }
  }
}

Answer 5

I found one way to do this without defining a function:

jq '.Resources | to_entries[] | select(.value.Type == "AWS::EC2::InternetGateway")|[{key: .key, value: .value}]|from_entries' example.json

{
  "ig001": {
    "Type": "AWS::EC2::InternetGateway",
    "Properties": {
      "Tags": [
        {
          "Key": "Name",
          "Value": "ig001"
        }
      ]
    }
  }
}