AWS lambda: Execute function on timeout

Question

I am developing a lambda function that migrates logs from an SFTP server to an S3 bucket.

Due to the size of the logs, the function sometimes is timing out - even though I have set the maximum timeout of 15 minutes.

try:
   logger.info(f'Migrating {log_name}... ')
   transfer_to_s3(log_name, sftp)
   logger.info(f'{log_name} was migrated succesfully ')

If transfer_to_s3() fails due to timeout logger.info(f'{log_name} was migrated succesfully') line won't be executed.

I want to ensure that in this scenario, I will somehow know that a log was not migrated due to timeout.

Is there a way to force lambda to perform an action, before exiting, in the case of a timeout?

Answer 1

Probably a better way would be to use SQS for that:

Logo info ---> SQS queue ---> Lambda function

If lambda successful moves the files, it removes the log info from SQS queue. If it fails, the log info persists in the SQS queue (or goes to DLQ for special handling), so the next lambda invocation can handle it.