Is there a way to incrementally build container images with AWS SAM?

Question

I have created a Lambda function using AWS SAM CLI which is deployed as a container image. Problem is the requirements are downloaded every time I make a small change in the code(app.py) and run sam build . The reason can be undestood from the Dockerfile below.

Dockerfile

FROM public.ecr.aws/lambda/python:3.8

COPY app.py requirements.txt ./
COPY models /opt/ml/models

RUN python3.8 -m pip install -r requirements.txt -t .

CMD ["app.lambda_handler"]

It downloads the requirements every time I run sam build .

I also came across a thread on github to use --cached option but it is not working if we use container image. https://github.com/aws/aws-sam-cli/issues/805

template.yml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Globals:
  Function:
    Timeout: 50
    MemorySize: 5000

Resources:
  InferenceFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      PackageType: Image
      Architectures:
        - x86_64
    Metadata:
      Dockerfile: Dockerfile
      DockerContext: ./app
      DockerTag: python3.8-v1

The dependency is tensorflow 2.8.0 which is over 200MB and I cannot change to any other options like tensorflow-lite.

Answer 1

With the way docker caching works, everything after your COPY statements is invalidated in cache (assuming changing). The way dependencies are often retained in cache is by only adding what is necessary to install dependencies, installing them, and then only adding your service code once dependencies are installed. In the example below, the pip install will only run more than once if requirements.txt changes.

FROM public.ecr.aws/lambda/python:3.8

COPY requirements.txt ./
RUN python3.8 -m pip install -r requirements.txt -t .

COPY app.py ./
COPY models /opt/ml/models

CMD ["app.lambda_handler"]