[英]Multithread queue of jobs
I have a queue of jobs which can be populated by multiple threads ( ConcurrentQueue<MyJob>
). 我有一个可由多个线程( ConcurrentQueue<MyJob>
)填充的作业队列。 I need to implement continuous execution of this jobs asynchronously(not by main thread), but only by one thread at the same time. 我需要异步执行此作业的连续执行(不是通过主线程),而只能同时执行一个线程。 I've tried something like this: 我已经尝试过这样的事情:
public class ConcurrentLoop {
private static ConcurrentQueue<MyJob> _concurrentQueue = new ConcurrentQueue<MyJob>();
private static Task _currentTask;
private static object _lock = new object();
public static void QueueJob(Job job)
{
_concurrentQueue.Enqueue(job);
checkLoop();
}
private static void checkLoop()
{
if ( _currentTask == null || _currentTask.IsCompleted )
{
lock (_lock)
{
if ( _currentTask == null || _currentTask.IsCompleted )
{
_currentTask = Task.Run(() =>
{
MyJob current;
while( _concurrentQueue.TryDequeue( out current ) )
//Do something
});
}
}
}
}
}
This code in my opinion have a problem: if task finnishing to execute( TryDequeue
returns false but task have not been marked as completed yet) and in this moment i get a new job, it will not be executed. 我认为这段代码有一个问题:如果要完成的任务完成( TryDequeue
返回false,但任务尚未标记为已完成),并且此刻我得到了一份新工作,它将无法执行。 Am i right? 我对吗? If so, how to fix this 如果是这样,如何解决此问题
Your problem statement looks like a producer-consumer problem, with a caveat that you only want a single consumer. 您的问题陈述看起来像生产者-消费者问题,但有一个警告,您只需要一个消费者。
There is no need to reimplement such functionality manually. 无需手动重新实现此类功能。 Instead, I suggest to use BlockingCollection
-- internally it uses ConcurrentQueue
and a separate thread for the consumption. 相反,我建议使用BlockingCollection
-在内部,它使用ConcurrentQueue
和一个单独的线程进行消耗。 Note, that this may or may not be suitable for your use case. 请注意,这可能适合您的用例,也可能不适合。
Something like: 就像是:
_blockingCollection = new BlockingCollection<your type>(); // you may want to create bounded or unbounded collection
_consumingThread = new Thread(() =>
{
foreach (var workItem in _blockingCollection.GetConsumingEnumerable()) // blocks when there is no more work to do, continues whenever a new item is added.
{
// do work with workItem
}
});
_consumingThread.Start();
Multiple producers (tasks or threads) can add work items to the _blockingCollection
no problem, and no need to worry about synchronizing producers/consumer. 多个生产者(任务或线程)可以将工作项添加到_blockingCollection
没有问题,也不必担心同步生产者/消费者。
When you are done with producing task, call _blockingCollection.CompleteAdding()
(this method is not thread safe, so it is advised to stop all producers beforehand). 完成生产任务后,请调用_blockingCollection.CompleteAdding()
(此方法不是线程安全的,因此建议事先停止所有生产者)。 Probably, you should also do _consumingThread.Join()
somewhere to terminate your consuming thread. 也许,您还应该在某处执行_consumingThread.Join()
来终止您的使用线程。
I would use Microsoft's Reactive Framework Team's Reactive Extensions (NuGet "System.Reactive") for this. 我将为此使用Microsoft的Reactive Framework Team的Reactive Extensions(NuGet“ System.Reactive”)。 It's a lovely abstraction. 这是一个可爱的抽象。
public class ConcurrentLoop
{
private static Subject<MyJob> _jobs = new Subject<MyJob>();
private static IDisposable _subscription =
_jobs
.Synchronize()
.ObserveOn(Scheduler.Default)
.Subscribe(job =>
{
//Do something
});
public static void QueueJob(MyJob job)
{
_jobs.OnNext(job);
}
}
This nicely synchronizes all incoming jobs into a single stream and pushes the execution on to Scheduler.Default
(which is basically the thread-pool), but because it has serialized all input only one can happen at a time. 这样可以很好地将所有传入的作业同步到一个流中,并将执行推送到Scheduler.Default
(基本上是线程池),但是因为它已经序列化了所有输入,一次只能发生一次。 The nice thing about this is that it releases the thread if there is a significant gap between the values. 这样做的好处是,如果两个值之间存在较大差距,它将释放线程。 It's a very lean solution. 这是一个非常精简的解决方案。
To clean up you just need call either _jobs.OnCompleted();
要清理,只需要调用_jobs.OnCompleted();
or _subscription.Dispose();
或_subscription.Dispose();
. 。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.