Azure.Messaging.ServiceBus SendMessageAsync timing out every 2 min?

Question

We are currently using v3 azure functions in a k8s cluster to send multiple messages to different queues in a service bus.

Here's the code we're using:

await using ServiceBusClient client = new ServiceBusClient(_configuration["AzureWebJobsServiceBus"]);

ServiceBusSender sender = client.CreateSender(queueName);
ServiceBusMessage message = new ServiceBusMessage(response);

await sender.SendMessageAsync(message);

For some reason we can observe a pattern where every ~2 min our.SendMessageAsync() seems to freeze and eventually timeout after 2 mins.

If we increase the amount of functions running in parallel, we would observe a higher peak, but the timeouts would occur at the same intervals.

The failed method from AppInsight:

Azure.Messaging.ServiceBus.Amqp.AmqpConnectionScope+d__57.MoveNext

We believe that when we call.SendMessageAsync, a connection to the queue is attempted, but eventually times out. We are aware of Azure Service Bus quotas and are currently running on the standard tier. When we look at the metrics of our service bus, nothing seems to be in the red.

After a thorough research online, we can't seem to find any information anywhere (or anyone else) that has the same problem as we do.

Packages being used:

• Azure.Messaging.ServiceBus 7.1.2
• Microsoft.NET.Sdk.Functions 3.0.11

Any chance we can get pointed towards a solution or a cause to our problem?

Answer 1

One thing you can try is sharing the ServiceBusClient between Azure Function instances:

private static ServiceBusClient _client = new ServiceBusClient(Environment.GetEnvironmentVariable("AzureWebJobsServiceBus"));

[FunctionName("HttpTriggerCSharp")]
public static async Task<IActionResult> Run(
    [HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)]
    HttpRequest req, ILogger log)
{
    log.LogInformation("C# HTTP trigger function processed a request.");

    ServiceBusSender sender = _client.CreateSender(Environment.GetEnvironmentVariable("QueueName");
    ServiceBusMessage message = new ServiceBusMessage(response);

    await sender.SendMessageAsync(message);
    
    return new OkObjectResult();
}

More info:

https://docs.microsoft.com/en-us/azure/architecture/antipatterns/improper-instantiation/

https://youtu.be/XO7TjQOQB_A?t=2450

Answer 2

You are likely running into the 5000 connection limit mentioned here . Each ServiceBusClient instance uses its own AMQP connection.