AWS Lambda Firing twice

Question

I have a lambda that is using an S3 trigger (PUT) when a .zip file is created. The lambda is unzipping the content to a folder in the same S3 (usually csvs) then moves the file to a PROCESSED folder and removes the .zip so it won't trigger again. In rare times, however, something weird is happening...the functions seems to have processed the same file twice...looking at versions, one.zip file was create but 2 x each content was extracted and the file .zip was 2 x deleted and 2 times renamed. So the first thing I tries was to look into CloudWatch but there is only one set of logs there. I also checked the function and there's no way it would just process the same file twice in one instance.

Answer 1

For a synchronous event trigger such as S3, this can happen on a very rare occasion from AWS. It is relevant in cases where the scale is relatively large (at least millions, if not billions).

A few things to consider:

1. First look if it's the same file - it might be a file re-upload? or maybe a delete event? Printing out the event into the logs might clarify that.
2. Look at the event to see if it has the same invocation ID, and the same event ID coming from S3. It it's indeed true it's that rare occasion.
3. Make your function idempotent - in short, it refers to the ability to identify repeated events and prevent duplicated, inconsistent, or lost data. You can read more here .
4. Make sure there are no issues or exceptions with the function.