[英]How to speed up Azure Storage Queue
I have tried everything I can think of to increase the speed of inserts. 我已经尝试了一切我能想到的提高插入速度的方法。 Which is really only a couple of things with no improvement.
这实际上只是一些没有改进的事情。
I have chunks of identifiers (Int64) that I need to send to a queue so that my multiple worker roles can work on it without having to worry about concurrency. 我有一些标识符(Int64)需要发送到队列,以便我的多个工作者角色可以在其上工作,而不必担心并发。
I have tried a foreach
loop (both with .ToString()
and BitConverter.GetBytes()
): 我尝试了一个
foreach
循环(使用.ToString()
和BitConverter.GetBytes()
):
foreach(long id in ids) {
queue.AddMessage(new CloudQueueMessage(id.ToString() /*BitConverter.GetBytes(id)*/));
}
And a Parallel .ForAll<T>()
: 并行
.ForAll<T>()
:
ids.AsParallel().ForAll(id => queue.AddMessage(new CloudMessage(id.ToString())));
Both from local and a WorkerRole inside the same data center, the inserts max out at 5 per second, and average 4.73 per second. 无论是本地还是同一数据中心内的WorkerRole,插入最大值为每秒5次,平均每秒4.73次。
Am I doing something wrong? 难道我做错了什么?
Try disabling Nagle on the tcp stack, as this buffers small packets, resulting in upwards of 1/2-second delay shipping your content. 尝试在tcp堆栈上禁用Nagle,因为这会缓冲小数据包,导致内容传输延迟超过1/2秒。 Put this in your role start code:
把它放在你的角色开始代码中:
ServicePointManager.UseNagleAlgorithm = false;
I used a different approach to pump 1500 messages per second from my laptop in a corporate network and hit the 2000 message limit on a VM co-located with the storage account. 我使用了一种不同的方法从公司网络中的笔记本电脑每秒输送1500条消息,并在与存储帐户位于同一位置的VM上达到2000条消息限制。
I used an async parallel partitioner in combination with some tuning of the default connection limit and partition count. 我使用了异步并行分区器,并结合了默认连接限制和分区计数的一些调整。
ServicePointManager.DefaultConnectionLimit = 1000;
public static async Task SendMessagesAsync(CloudQueue queue, IEnumerable<string> messages)
{
await Task.WhenAll(
from partition in Partitioner.Create(messages).GetPartitions(500)
select Task.Run(async delegate
{
using (partition)
while (partition.MoveNext())
await queue.AddMessageAsync(new CloudQueueMessage(partition.Current));
}));
}
Nagle on or off had no impact on performance. Nagle开启或关闭对性能没有影响。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.