AWS Lambda中的S3客户端初始化速度缓慢

Question

I'm currently working on an AWS Lambda function written in Java. It needs to fetch objects from S3 and I have therefore set up an IAM role and am building an S3 client in the Lambda's handler:

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;

public class Example implements RequestHandler<Void, Void> {

    @Override
    public Void handleRequest(Void nothing, Context ctx) {
        long start = System.currentTimeMillis();
        AmazonS3 amazonS3 = AmazonS3ClientBuilder.defaultClient();
        ctx.getLogger().log("Creating S3 client took " + (System.currentTimeMillis() - start) + "ms");
        ...
        return null;
    }
}

Nevertheless, using the AmazonS3ClientBuilder is very slow, the log statement prints out the following timings when allocating 192MB to the function:

Creating S3 client took 13541ms
Creating S3 client took 16482ms
Creating S3 client took 13617ms
Creating S3 client took 16380ms

Even when bumping memory right up to 3008MB to get maximum processing power (as AWS allocates CPU power proportional to the memory for Lambdas), it still takes between 1 and 2 seconds to get the client:

Creating S3 client took 1413ms
Creating S3 client took 1170ms
Creating S3 client took 1528ms
Creating S3 client took 1394ms

These timings were recorded in cold start scenarios and I am caching the AmazonS3 instance for subsequent requests, but it seems pretty extreme that just building an S3 client can take over 16 seconds on a non warm Lambda.

Am I misusing the AmazonS3ClientBuilder , possibly by not overriding some default values, leading to poor performance? How can client initialisation be sped up?

Answer 1

AWS clients are considered thread safe and can be safely used by multiple requests at the same time. You should be able to create the client as a member variable and reuse the same client for each request. This should save considerable time since the default client makes a few aws calls each time it is created to find the region and credentials.

Answer 2

The AWS SDK for Java 2.0 was released in November. It's basically a rewrite of version 1.x and it seems to have improved performance quite a bit. Migrating the code from the question to use the new SDK would give us something similar to the following:

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import software.amazon.awssdk.services.s3.S3Client;

public class Example implements RequestHandler<Void, Void> {

    @Override
    public Void handleRequest(Void nothing, Context ctx) {
        long start = System.currentTimeMillis();
        S3Client s3Client= S3Client.create();
        ctx.getLogger().log("Creating S3 client took " + (System.currentTimeMillis() - start) + "ms");
        ...
        return null;
    }
}

When allocating 192MB to the function:

Creating S3 client took 9380ms
Creating S3 client took 9719ms
Creating S3 client took 10098ms
Creating S3 client took 9519ms

When allocating 3008MB to the function:

Creating S3 client took 884ms
Creating S3 client took 873ms
Creating S3 client took 886ms
Creating S3 client took 877ms

Based on these rough figures, S3 client creation time in cold start scenarios has decreased by over a third when using version 2 of the SDK.

Answer 3

When running a serverless function, it will stay active (aka, hot) as long as you're running it. Your container stays alive, ready and waiting for execution.

After a period of inactivity, your cloud provider will drop the container, and your function will become inactive, (aka, cold).

A cold start happens when you execute an inactive function. The delay comes from your cloud provider provisioning your selected runtime container and then running your function.

You could fix this by keeping your functions 'warm'. One way to do this would be to add a cronjob that pings your function every now and then. There is a plugin available that does just that called serverless-plugin-warmup , and I am sure there are many more just like that.