Change AWS Lambda Kinesis stream polling frequency

Question

I want to change Kinesis stream polling frequency of AWS Lambda function. I was going through this article: https://docs.aws.amazon.com/lambda/latest/dg/with-kinesis.html but, no luck. The only information it conveys is AWS Lambda then polls the stream periodically (once per second) for new records .

I was also looking for answers in threads, but no luck: https://forums.aws.amazon.com/thread.jspa?threadID=229037

There is another option though, which can be used if desired frequency is required: https://docs.aws.amazon.com/lambda/latest/dg/with-scheduled-events.html

So, my question is, can we decrease AWS Lambda's polling frequency to, lets say 1-2 mins? Or do we have to go with AWS Lambda with Scheduled Events ?

Answer 1

As far as I know there is now way to decrease the polling frequency if you are using an event source mapping.

These are all the settings you can set (source: https://docs.aws.amazon.com/de_de/lambda/latest/dg/API_CreateEventSourceMapping.html ):

{ "BatchSize": number, "Enabled": boolean, "EventSourceArn": "string", "FunctionName": "string", "StartingPosition": "string", "StartingPositionTimestamp": number }

So going with a scheduled event seems to be the only feasible option. An alternative would be to let the lambda function sleep before exiting so it will only poll again after a desired time. But of course this means you are paying for that.. So this is probably not desired.

Answer 2

I haven't seen a way to decrease the polling frequency, but you can have the same effect as if polling frequency decreased by increasing the MaximumBatchingWindowInSeconds parameter.

Reference: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-property-function-kinesis.html#sam-function-kinesis-maximumbatchingwindowinseconds

Let's say you have new records arriving on average at 1 record/s. Regardless of BatchSize, your lambda might trigger every second as it polls once every second. But if you increase your BatchSize to let's say, 60 and the MaximumBatchingWindowInSeconds to 60, then your lambda only invokes on average once every minute, as if you've changed polling frequency to once per minute.