Terraform AWS Security Group settings to allow all ports between all VMS

Question

I'm kinda stumped. I have a custom service (named bacalhau) running on each of three machines in a security group on 54545. I also have SSH running on all machines. Here's what my terraform looks like:

resource "aws_security_group" "allow_ssh_and_bacalhau" {
  vpc_id      = aws_vpc.bacalhau_vpc.id
  name        = "allow_ssh_and_bacalhau"
  description = "security group that allows ssh and bacalhau and all egress traffic"

}
resource "aws_security_group_rule" "egress_all" {
  type              = "egress"
  from_port         = 0
  to_port           = 0
  protocol          = "-1"
  cidr_blocks       = ["0.0.0.0/0"]
  security_group_id = aws_security_group.allow_ssh_and_bacalhau.id
}


resource "aws_security_group_rule" "ingress_ssh" {
  type              = "ingress"
  from_port         = 22
  to_port           = 22
  protocol          = "-1"
  cidr_blocks       = ["0.0.0.0/0"]
  security_group_id = aws_security_group.allow_ssh_and_bacalhau.id
}

resource "aws_security_group_rule" "ingress_bacalhau" {
  type              = "ingress"
  from_port         = 54545
  to_port           = 54545
  protocol          = "-1"
  self              = true
  security_group_id = aws_security_group.allow_ssh_and_bacalhau.id
}

SSH works fine - including inter-traffic between machines, but the bacalhau (54545) service doesn't show up.

Eg

ubuntu@ip-10-0-1-219:~$ nmap ec2-18-202-245-138.eu-west-1.compute.amazonaws.com
Starting Nmap 7.80 ( https://nmap.org ) at 2022-03-19 18:07 UTC
Nmap scan report for ec2-18-202-245-138.eu-west-1.compute.amazonaws.com (10.0.1.237)
Host is up (0.0022s latency).
rDNS record for 10.0.1.237: ip-10-0-1-237.eu-west-1.compute.internal
Not shown: 999 closed ports
PORT   STATE SERVICE
22/tcp open  ssh

Nmap done: 1 IP address (1 host up) scanned in 0.08 seconds
ubuntu@ip-10-0-1-219:~$

EDIT The nmap is running from a VM inside the security group.

Am I doing something wrong? Is this a security group, vpc, or ec2 mistake? I can access the service from the node itself through localhost loop back.

EDIT 2 Confirmed this is a security group issue - I turned on accepting inbound from everywhere [0.0.0.0/0] and it worked fine.

Answer 1

For ingress_bacalhau security group rule you have set argument of self = true . This will allow traffic only from another instance which also has the allow_ssh_and_bacalhau security group attached.

In contrast, for SSH with ingress_ssh rule you allow traffic from the whole inte.net ( cidr_blocks = ["0.0.0.0/0"] )

You do not specify clearly from which instance was the nmap scan executed. If this instance does not have the allow_ssh_and_bacalhau security group attached, then the traffic wont be allowed from it.