[英]How to execute a lot of durable functions triggered by Azure Queue?
If briefly, our task is to process a lot of input messages.简而言之,我们的任务是处理大量输入消息。
To solve this we decided to use Azure Queue Storage and Azure Functions.为了解决这个问题,我们决定使用 Azure 队列存储和 Azure Functions。 We have Azure Functions structure similar to the following code:
我们有类似于以下代码的 Azure Functions 结构:
Queue triggered function队列触发功能
[FunctionName("MessageControllerExecutor")]
public static async void Run(
[QueueTrigger(QUEUE_NAME, Connection = QUEUE_CONNECTION_NAME)]string queueMessage,
[OrchestrationClient] DurableOrchestrationClient client,
TraceWriter log)
{
await client.StartNewAsync("MessageController", queueMessage);
}
Durable function耐用功能
[FunctionName("MessageController")]
public static async void Run(
[OrchestrationTrigger] DurableOrchestrationContext context,
TraceWriter log)
{
if (!context.IsReplaying) log.Warning("MessageController started");
var function1ResultTask = context.CallActivityAsync<ResultMessage>("Function_1", new InputMessage());
var function2ResultTask = context.CallActivityAsync<ResultMessage>("Function_2", new InputMessage());
await Task.WhenAll(function1ResultTask, function2ResultTask);
// process Function_1 and Function_2 results
// ...
}
Simple activity function sample简单活动函数示例
[FunctionName("Function_1")]
public static ResultMessage Run(
[ActivityTrigger] DurableActivityContext activityContext,
TraceWriter log)
{
var msg = activityContext.GetInput<InputMessage>();
int time = new Random().Next(1, 3);
Thread.Sleep(time * 1000);
return new ResultMessage()
{
Payload = $"Function_1 slept for {time} sec"
};
}
MessageControllerExecutor
triggered when a new item is received in a queue. MessageControllerExecutor
在队列中接收到新项目时触发。 MessageController
is a Durable Function that uses a few simple activity functions to process each message. MessageController
是一个 Durable Function,它使用一些简单的活动函数来处理每条消息。
When we push messages to the queue, the MessageControllerExecutor
function starts immediately and asynchronously fires the MessageController
and passes the message, so this works as expected.当我们将消息推送到队列时,
MessageControllerExecutor
函数会立即启动并异步触发MessageController
并传递消息,因此这按预期工作。
But we are faced with the problem.但是我们面临着这个问题。 Not all
MessageController
function instances run.并非所有
MessageController
函数实例都运行。
For example, we pushed 100 messages into the queue, but only about 10-20% of the messages were processed by MessageController
.例如,我们将 100 条消息推送到队列中,但只有大约 10-20% 的消息被
MessageController
处理。
Some messages were not processed or were processed with a long delay.某些消息未处理或处理延迟很长时间。 It looks like durable functions failed to start б, though no exceptions were thrown.
看起来持久函数未能启动 б,尽管没有抛出异常。
We have a few questions:我们有几个问题:
You did elide some code from your orchestration trigger in the question above for the purposes of brevity which I understand, but what exactly are you doing there after the await Task.WhenAll(...)
?为了我理解的简洁起见,您确实从上面的问题中的编排触发器中删除了一些代码,但是在
await Task.WhenAll(...)
之后您到底在做什么? If it includes any kind of significant processing you should really be farming that out to a third action function (eg Function_3
) to do and then simply returning the results from the orchestration function.如果它包含任何类型的重要处理,您应该真正将其分配给第三个操作函数(例如
Function_3
)来执行,然后简单地从编排函数返回结果。
Update: I just noticed your functions are defined as async void
.更新:我刚刚注意到你的函数被定义为
async void
。 If I had to guess, this would actually cause a problem for the runtime.如果我不得不猜测,这实际上会导致运行时出现问题。 Can you try changing it to
async Task
and see if your problem goes away?您可以尝试将其更改为
async Task
并查看您的问题是否消失吗? As a general rule defining methods as async void
is frowned upon in .NET .作为一般规则,在 .NET 中
async void
将方法定义为async void
。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.