[英]identifiing the simultaneous tasks in a TPL dataflow
I have 1000 elements in a TPL dataflow block, each element will call external webservices. 我在TPL数据流块中有1000个元素,每个元素都会调用外部Web服务。
the web service supports a maximum of 10 simultaneous calls, which is easily achieved using: Web服务最多支持10个同时呼叫,这可以通过以下方式轻松实现:
new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = 10
...
}
The web service requires each call to have a unique id passed which distinguises it from the other simultaneous calls. Web服务要求每个调用都有一个唯一的id,它将其与其他同时调用区分开来。 In theory this should be a guid, but in practise the 11th GUID will fail - because the throttling mechanism on the server is slow to recognise that the first call is finished. 理论上这应该是一个guid,但实际上第11个GUID将失败 - 因为服务器上的限制机制很难识别第一个调用已完成。
The vendor suggests we recycle the guids, keeping 10 in active use. 供应商建议我们回收guids,保持10个正在使用。
I intend to have an array of GUIDS, each task will use (Interlocked.Increment(ref COUNTER) % 10 ) as the array index 我打算有一个GUID数组,每个任务都会使用(Interlocked.Increment(ref COUNTER)%10)作为数组索引
EDIT : I just realised this won't work! 编辑:我刚刚意识到这不起作用! It assumes tasks will complete in order which they may not I could implement this as a queue of IDs where each task borrows and returns one, but the question still stands, is there a an easier, pre bulit thread-safe way to do this? 它假设任务将按顺序完成,我们可能无法将其作为ID队列实现,其中每个任务借用并返回一个,但问题仍然存在,是否有一个更容易的,预备好的线程安全的方法来做到这一点?
(there will never be enough calls for COUNTER to overflow) (永远不会有足够的呼叫让COUNTER溢出)
But I've been surprised a number of times by C# (I'm new to .net) that I am implementing something that already exists. 但是我很多次对C#(我是.net的新手)感到惊讶,我正在实现已经存在的东西。
Is there a better thread-safe way for each task to recycle from a pool of ids? 是否有更好的线程安全方式让每个任务从一个id池中回收?
Creating resource pools is the exact situation System.Collections.ConcurrentBag<T>
is useful for. 创建资源池的确切情况是System.Collections.ConcurrentBag<T>
非常有用。 Wrap it up in a BlockingCollection<T>
to make the code easier. 将其包装在BlockingCollection<T>
以使代码更容易。
class Example
{
private readonly BlockingCollection<Guid> _guidPool;
private readonly TransformBlock<Foo, Bar> _transform;
public Example(int concurrentLimit)
{
_guidPool = new BlockingCollection<Guid>(new ConcurrentBag<Guid>(), concurrentLimit)
for(int i = 0: i < concurrentLimit; i++)
{
_guidPool.Add(Guid.NewGuid());
}
_transform = new TransformBlock<Foo, Bar>(() => SomeAction,
new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = concurrentLimit
//...
});
//...
}
private async Task<Bar> SomeAction(Foo foo)
{
var id= _guidPool.Take();
try
{
//...
}
finally
{
_guidPool.Add(id);
}
}
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.