耐用 Function 延迟触发

Question

I'm currently facing a weird issue.

Randomly (I guess) my azure durable function invocation is triggered with delay >10min . My understanding is that there's something wrong with the lease for control queue.

I'm on Consumption Plan . So i'm wondering if the scale-in/out mechanism is working properly with my durable function . My feeling is that a host instance takes the lease then goes into drain mode -> recycling etc. and keeps the lease during 10min before releasing.

My feeling is that it's happening after a period of inactivity.

Have you ever seen such behavior?

Answer 1

I found the similar recent issue # 1148771 reported on January 2023 in MS Q&A Forum of Azure Functions where the user is experiencing delay in durable functions and the hosting model is Consumption Plan during Orchestration start.

This case still is in investigation by the Microsoft Support team and mentioned some of the reasons such as:

The lease for control queue is held by previous function instance which is supposed to be recycled.

• For the basic checks, make sure you have the latest versions of all the packages used in the function code.

• Also, in the same Q&A Forum, mentioned the scenario what happened for causing the issue based on the timestamps, orchestration Ids provided. If your scenario is similar, then you can track the discussion for the solution.

• If it is product issue, then you could raise a ticket in azure-functions-durable-extensions repository of the GitHub.

Refer to the GitHub Issue # 606 and this MS Doc troubleshooting steps of Orchestration start delays in Azure Durable Functions.

Answer 2

Today, I've received the update from the product team that the issue is caused by the current Lease Management logic used by Durable Function. It could happen when an instance was shut down.

This issue is specific to Azure Storage backend.

This issue has been existing for a while. We're trying to refactor the current Lease Management logic with hope to fix such issue, but no clear ETA yet. You can subscribe to Orchestrations freezing mid-execution · Issue #2207 · Azure/azure-functions-durable-extension (github.com) to get the notification when the fix is ready.