Slow performance with Durable Functions running on an App Service Plan?

Question

I am testing out Durable Functions on an App Service Plan, because some of my activities will run for +20 minutes, and I don't want to pay for Premium right now.

The workflow is essentially this:

• Get a list of updates (IDs, usually around 20-30k) from a third party service
• For all IDs, call another activity func with one ID as input (in parallel)
• Summarize all the results and return this from the orchestrator

This is running very slow. This workflow is currently running as a WebJob in prod. For one of our integrations, It loops through each ID sequentially and takes around 20 minutes to complete. With Durable Functions, it is using 40 minutes. How can this be the case? Shouldn't it run faster since it is handling each ID in parallell? I get that there is an overhead with all the queues in the background, but will it really be this slow?

Will increasing the amount of activities it can run help? I see that the CPU% for the App Plan is on 100% when running this workflow, so I'm not sure.

Running on the B2 plan.

If anyone has any tips or experiences using DF on app plan, I would appreciate any feedback!

Updated with host.json:

{
  "version": "2.0",
  "logging": {
    "logLevel": {
      "default": "Information",
      "Host.Results": "Information",
      "Function": "Information",
      "Host.Aggregator": "Information"
    }
  },
  "extensions": {
    "durableTask": {
      "maxConcurrentActivityFunctions": 200,
      "maxConcurrentOrchestratorFunctions": 100,
      "extendedSessionsEnabled": true,
      "extendedSessionIdleTimeoutInSeconds": 600,
      "controlQueueBatchSize": 1024,
      "controlQueueBufferThreshold": 512
    }
  }
}

Answer 1

It sounds like you're using fan in fan out pattern

Where the following corresponds to the sample code

• (F1) Get a list of updates (IDs, usually around 20-30k) from a third party service
• (F2) For all IDs, call another activity func with one ID as input (in parallel)
• (F3) Summarize all the results and return this from the orchestrator

Based on that assumption, I don't see increasing the amount of activities helping your situation but rather it's just a band-aid. Analyzing your telemetry to see where the CPU, I'm assuming in F2, is spending most of its time is what I would do. Also, have a look at Performance and scale in Durable Functions (Azure Functions) , specifically the Performance targets section just to ensure your current configuration isn't inadvertently causing bottlenecks.

If your summary (F3) isn't dependent on the batch of IDs that were processed, then stateless is another way of doing where F1 outputs IDs to a service bus queue and F2 is triggered off that service bus.