AWS Lambda function to read from S3 and write to Elastic Cache

Question

I have to two separate Lambda functions - one to read a file from a S3 bucket and write to memcache cluster. They work well individually. However, I am unable to 'merge' them together.

Firstly, the function to read from S3 works from 'No VPC' setting whereas, the function to write to Elastic Cache works only when the function and cluster are in the same VPC.

Secondly, the function to read from S3 had only the AmazonS3FullAccess policy applied. While I have now applied the AWSLambdaVPCAccessExecutionRole also, I am not sure, if this setting will work because of the VPC difference mentioned above.

Is AWS Step function the answer? How do I build a serverless application that reads a file from S3 and writes to Elastic Cache cluster?

Answer 1

You don't need step functions for this. Run the function in the VPC with your ElastiCache cluster. Either add an S3 endpoint to your VPC, or a NAT gateway. The S3 endpoint is the easiest solution. Then your function will have access to both ElastiCache and S3.

For the IAM role, you need to go into IAM and create a new role that has the permissions of AWSLambdaVPCAccessExecutionRole as well as the necessary S3 permissions. You can assign multiple policies to a single role if necessary. Then assign that role to the Lambda function.