AWS Lambda connecting to Internet

Question

TL;TR

I am trying to connect to inte.net from AWS Lambda, I have a private su.net with a NAT Gateway but still the function cannot connect to inte.net...

Full Question

So I am trying to access inte.net with my AWS Lambda function. I have tried both Java and NodeJS 4 with no luck.

I have a private VPC with a su.net: 10.0.10.0/24

As you can see I have added a rule to my NAT Gateway:

I configured my AWS Lambda like this:

Selecting that su.net (10.0.10.0) and with a security group that is open to everything (both inbound and outbound)

But yet when I try to download something from inte.net, the lambda times out:

'use strict';
console.log('Loading function');

var http = require("http");

exports.handler = (event, context, callback) => {
    //console.log('Received event:', JSON.stringify(event, null, 2));
    console.log('value1 =', event.key1);
    console.log('value2 =', event.key2);
    console.log('value3 =', event.key3);

    var options = {
      host: 'www.virgilio.it',
      port: 80,
      path: '/'
    };

    http.get(options, function(res) {
      console.log("Got response: " + res.statusCode);
    }).on('error', function(e) {
      console.log("Got error: " + e.message);
    });

    callback(null, event.key1);  // Echo back the first key value
    // callback('Something went wrong');
};

{ "errorMessage": "2016-05-10T10:11:46.936Z 79968883-1697-11e6-9e17-1f46a366f324 Task timed out after 55.00 seconds" }

Is this a bug?

Note: the same function works If I don't select my VPC

Answer 1

By default, a lambda function is not bounded to a VPC, which enables it to have internet access, but prevents it from accessing resources in a VPC, such as RDS instances.

If you attach the lambda to a VPC, you'll loose internet access, which prevents you from accessing resources such S3 and Dynamo, and from making HTTP requests.

If you need both, then I'll have to set up the VPC for internet access, which is a mess (hey AWS guys, if you have a well-defined process for it, please make it simple: turn it into a checkbox or button ;)

Create a new VPC

I find it's best to leave the default VPC alone, so you don't take the risk of breaking something that's already working in that VPC (in case you already have resources there), and also because you can use the default VPC as configuration reference in the future.

Use the wizard for creating the VPC.

Create the Route Tables

1. Name the first public-subnet (if it's not already there);
2. Name the second private-lambda . AWS support recommends having a separate subnet just for the lambda, and this Route Table is going to be attached to it.

Create the subnets

By default, when you create a VPC, it will create a public subnet for you. If you used default values, its name should be Public subnet . Leave it at that.

Now you are going to create the private subnets. Is recommended to have several private subnets for your Lambda if you want it to have high availability.

Each of these private subnets will be linked to the VPC you just created. Now, supposing you left the VPC IP as 10.0.0.0/16 , and that you run your resources in Virginia ( us-east-1 ), here is a template for creating six private subnets, each in a different availability zone (for high availability):

1. private-lambda-us-east-1a , availability zone us-east-1a , IP block 10.0.16.0/24
2. private-lambda-us-east-1b , availability zone us-east-1b , IP block 10.0.32.0/24
3. private-lambda-us-east-1c , availability zone us-east-1c , IP block 10.0.48.0/24
4. private-lambda-us-east-1d , availability zone us-east-1d , IP block 10.0.64.0/24
5. private-lambda-us-east-1e , availability zone us-east-1e , IP block 10.0.80.0/24
6. private-lambda-us-east-1f , availability zone us-east-1f , IP block 10.0.92.0/24

But you can see the pattern: - There's a 16 increment in the 3rd position of the IP block; - The names indicate the selected availability zone in your region.

Ensure Route Table vs Subnet associations

• Go to the Route Tables panel;
• Select the public-subnet table, review its associations and make sure it's associated to the Public Subnet;
• Select the private-lambda table, review its associations and make sure It's associated to all the private-lambda-* subnets you just created.

Create an Internet Gateway

Just create one and attach it to the VPC.

Configure the routes for the Public Subnet

In my case it came configured, but just make sure that the Route Table for your Public Subnet has an entry from 0.0.0.0/0 to your just-created Internet Gateway.

Create a NAT (network address translator)

Create a new NAT and select your Public Subnet. Allocate a new EIP.

Configure the routes for the Private Subnets

Ensure that the Route Table for your Private Subnets has an entry from 0.0.0.0/0 to your new NAT.

And with these steps, you should now have an Internet-enabled VPC.

---

Use Case: configuring a Lambda for internet and RDS access

Create a Security Group for the lambda

• New up a SG and configure Outbound -> All Trafic -> to 0.0.0.0/0 and ::/0

Modify the Security Group of your RDS instance to allow

• Inbound -> All trafic -> from the lambda SG

Configure the lambda

• Create a new lambda or select an existing one;
• Select your new VPC;
• Select all your private subnets ( private-lambda-* ) for high availability;
• Select your lambda Security Group.

And that's it. You should now have a lambda function that can access both VPC and Internet resources :)

Answer 2

I found the error, the NAT Gateway should be added to a public subnet (not a private one).
A public subnet is the subnet with the Internet Gatway route associated to 0.0.0.0/0

Answer 3

Since I faced the same issue adding a bit more clarity to the above answer -

1. Add NAT Gateway or NAT instance to public subnet (the one that has an 0.0.0.0/0 entry to internet gateway in corresponding (public) subnets route table)
2. Edit route table of private subnet (where you are running your lambda) to have an entry for 0.0.0.0/0 to the NAT gateway in the public subnet.
3. Make sure security group assigned to lambda allows outbound connections.

Answer 4

If you are to do this with Terraform, here are the resources that you will need to configure (in addition to Lambda resources):

• aws_vpc
• aws_subnet (public and private; multiple for high availability)
• aws_internet_gateway for outbound connection to the internet
• aws_route_table (x2 for public and private subnets)
• aws_route_table_association
• aws_eip (Elastic IP for the NAT Gateway)
• aws_nat_gateway
• aws_default_network_acl to manage the default NACL with Terraform
• aws_default_security_group to manage the default Security Group with Terraform
• aws_iam_role_policy_attachment to attach AWSLambdaVPCAccessExecutionRole policy

Deploy AWS Lambda to VPC with Terraform .

Answer 5

For the ease if you are starting use the VPC launch template on the VPC home page to deploy the VPC. This will create public subnet, private subnet, NAT GW, IGW and all the required routes to access internet from private subnet.

Answer 6

If you need both, then I'll have to set up the VPC for inte.net access, which is a mess (hey AWS guys, if you have a well-defined process for it, please make it simple: turn it into a checkbox or button;)

Your prayers have been heard and AWS now has an "Resources to create" -> "VPC, su.nets, etc." option when creating a new VPC where you can just choose the amount of su.nets, AZs, NAT Gateways etc. in the wizard and it creates everything you described for you: Create VPC

Answer 7

I had a similar problem when trying to access the Secrets Manager after adding the Lambda function to a VPC. On this thread it discusses a NAT solution but also using a VPC endpoint:
AWS Lambda access Secrets Manager from within VPC

More information:
Access an AWS service using an interface VPC endpoint