Regarding KInesis Firehose data stream to AWS Lambda

Question

Suppose live streaming is happening and data will be collected on Kinesis and process the data.

Let's say 10 seconds happened and it will trigger first lambda function and assume that lambda function will take only 5 second to process that data.

So,the first lambda function will process in 15 second.but in between assume that some live data streaming is occur at 12 sec from Kinesis and once again it will trigger Lambda function.

So,second lambda function will execute once first lambda function will complete or is it possible that both the lambda function will be working in parallel?

Can someone please clarify and if we have any document regarding this thing then please share the link.

I am not able to find regarding this thing.

Answer 1

Not so straightforward answer.

Generally speaking, it depends on how many shards you configured for the stream. Each shard will invoke a single lambda only, no parallel invocations on a stream. Hence everything is processed in order by that one function for that one shard.

But now (since 2019) you can set a parallelization factor, so that each shard can have up to 10 parallel functions operating on them.

Link here: https://aws.amazon.com/about-aws/whats-new/2019/11/aws-lambda-supports-parallelization-factor-for-kinesis-and-dynamodb-event-sources/

So in conclusion, by default it's one function per shard. Unless you set a Parallelization Factor, in which case you can have up to 10 parallel functions executing.

It's a bit confusing, but the link above will help explain.