GCP Pub/Sub 重试次数超过配置

Question

I have a Pub/Sub push subscription that is triggering Cloud Function. There is retry policy with expotential backoff and dead lettering enabled.

Configuration is next:

• Acknowledge deadline is 60s
• Exactly once delivery enabled
• Maximum delivery attempts is 5
• Miniumn backoff - 30s, maximum backoff - 600s

For testing purposes I configured Cloud Function to always return 500 error code. But instead of 5 retries and 1 event in dead letter topic there was:

• 6 retries and 2 events in dead letter topic
• 7 retries and 3 events in dead letter topic
• 10 retries and 6 events in dead letter topic

I can assume that dead letter topic received more events because there was more retries than expected. But I cannot understand why there was more retries than configured?

Answer 1

Pub/Sub has a built in At-least-once delivery system which will retry messages that were not acknowledged. In this case, after 600s have passed, the message you first sent becomes unacknowledged, thus Pub/Sub retries the message. It will keep retrying it for 600s until it reaches the messageRetentionDuration or you acknowledge it.