[英]Best Queue Consumer implementation in Java
Good day! 美好的一天!
I want to make an ExecutorService consumers for taking data from queue and working with it on server side. 我想让ExecutorService使用者从队列中获取数据并在服务器端使用它。 The idea is - I poll queue from time to time and if I see that it is not empty I start ExecutorService with N threads (lets say 5).
这个想法是-我不时轮询队列,如果我发现队列不为空,则使用N个线程(让我说5个)启动ExecutorService。 Then I w8 while queue will be empty and shutdown threads.
然后我w8而队列将为空并关闭线程。 And all again - poll queue for data.... Is this alg ok?
再说一遍-轮询队列中的数据...。 Or may be there are some ready implementations/frameworks for such task?
还是可能有一些准备就绪的实现/框架来完成此类任务?
I found this implementation of ConcurrentQueue cunsumers : 我发现了ConcurrentQueue cunsumers的此实现:
public class ConcurrentQueueClient implements Runnable {
private Queue<String> concurrentQueue;
public ConcurrentQueueClient(Queue concurrentQueue) {
this.concurrentQueue = concurrentQueue;
}
public void run() {
boolean stopCondition = (concurrentQueue.size() == 0);
while (!stopCondition) {
for (int i = 0; i < concurrentQueue.size(); i++) {
System.out.println("Client dequeue item "
+ concurrentQueue.poll());
}
stopCondition = (concurrentQueue.size() == 0);
}
System.out.println("Client thread exiting...");
}
}
and testing it in such way : 并以这种方式进行测试:
Queue<String> queue = new ConcurrentLinkedQueue<String>();
ExecutorService consumers = null;
while(true) {
if(queue.size() != 0) {
consumers = Executors.newFixedThreadPool(100);
for (int i = 0; i < 5; i++) {
ConcurrentQueueClient client = new ConcurrentQueueClient(queue);
consumers.execute(client);
}
}
while (queue.size() != 0) {
try {
Thread.sleep(1500);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
consumers.shutdown();
try {
consumers.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
Start over. 重来。 Wrap your strings in callable or runnable, and queue those to the executor service.
将字符串包装成可调用或可运行的字符串,然后将它们排队到执行程序服务中。
If you have a finite set of data to process, then it's ok to have the main thread calling consumer.shutdown() and consumer.awaitTermination(...) as before, but no sleep loop. 如果您要处理的数据有限,那么可以像以前一样让主线程调用consumer.shutdown()和consumer.awaitTermination(...),但是没有睡眠循环。 If you are going to process indefinitely from the queue, then no shutdown() until the service is.
如果您要从队列中无限期地进行处理,则在服务停止之前,不执行shutdown()。
You will face memory issues too if you don't have a limited blocking queue (nothing to block queue.put()). 如果没有有限的阻塞队列(什么也不会阻塞queue.put()),您也会遇到内存问题。 An ArrayBlockingQueue can be given to the executor service on creation (see ThreadPoolExecutor(...) )
可以在创建时将ArrayBlockingQueue赋予执行者服务(请参阅ThreadPoolExecutor(...))
Executor service's threads are doing that check (queue.take()) of the tasks queue by design. 执行器服务的线程正在按照设计检查(queue.take())任务队列。 Try to avoid polling, it waste CPU.
尝试避免轮询,这会浪费CPU。 Always try to wait/notify (or await/signal) on conditions from reentrantlocks (which is all taken care of for you in the executor service code)
始终尝试在reentrantlocks的条件下等待/通知(或等待/发出信号)(执行程序服务代码中的所有内容都由您来处理)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.