Requests to Lambda through ALB behind VPC are extremely slow

Question

I am working on exposing a lambda to the public.net with the lambda residing behind an already existing VPC (so I can later on limit the IP range of incoming requests using a security group).

To test that everything works, I set up a small lambda that simply prints hello world. I am running into a problem where the connection is extremely slow. The lambda executes in less than a milisecond, but each CURL to the endpoint run extremely slow.

Using curl for diagnostics I have found that:

curl -kso /dev/null my-alb-url -w "==============\n\n
| dnslookup: %{time_namelookup}\n
| connect: %{time_connect}\n
| appconnect: %{time_appconnect}\n
| pretransfer: %{time_pretransfer}\n
| starttransfer: %{time_starttransfer}\n
| total: %{time_total}\n
| size: %{size_download}\n
| HTTPCode=%{http_code}\n\n"
==============


| dnslookup: 0.061576

| connect: 75.256759

| appconnect: 0.000000

| pretransfer: 75.257615

| starttransfer: 75.794737

| total: 75.795154

| size: 28

| HTTPCode=200

The load balancer:

1. is connected to two availability zones that are both public facing
2. forward to a target group only containing my lambda
3. is linked with a security group that has enabled all inbound and outbound traffic

To make things more confusing, this is not an issue on every request but it is seemingly random.

What would be the best way to debug this issue?

Answer 1

I managed to resolve it but I am not entirely sure how.

I triple checked the su.net setup and just redid the setup again. 10 minutes later and now it works as intended. My suspicion is that one of the AZs linked to the wrong su.net.