Client side public key encryption

Question

We are building a mobile app that has the following functions

1. Capture user information (text, audio) while not connected to the internet
2. Store the information on the device, encrypted using a server-side public key
3. When internet connectivity is available, upload to s3 using pre-signed URL
4. Once the file is available in s3 another background process(?) decrypts it using the server-side private of the above public-key and copy to another bucket

Due to regulatory requirements, files on the device need to be encrypted at rest while not connected to the internet. Otherwise s3 encryption client ( AmazonS3EncryptionClient ) could be used for this purpose to encrypt on the fly when uploading to.

My question is on step 4. Is it possible to use an s3 trigger lambda function for this step? I imagine the limit of available disk size for lambda will mean copying file from s3 to lambda local directory will not work for large files (the files can be few gigabytes large). What other options are available (preferably serverless)? What about streaming s3 object and decrypt in-memory?

Answer 1

Is it possible to use an s3 trigger lambda function for this step? .. What about streaming s3 object and decrypt in-memory?

You may read-decrypt-write on s3 as a stream, then the lambda limitation will at the execution time (900s = 15 minutes). If you could process the file in 15 minutes, using lamba may be good idea. IMHO you can process a lot of GB in 15 minutes, but there's always the 15 min. limit (it was 5 minutes some time ago).

What other options are available

If you want to process the files without any processing-time limit, I'd suggest to have the s3 lambda send a message to an AWS Job Queue