![](/img/trans.png)
[英]RabbitMQ - only one queue, with multiple consumers receiving different messages
[英]Multiple consumers on one RabbitMQ queue
我正在遵循 RabbitMQ 的本指南: https : //www.rabbitmq.com/tutorials/tutorial-two-java.html 。 我想用一個隊列上的多個線程來模擬這個功能。
如果我在啟動發送器之前啟動接收器,它會按預期工作,如下所示:
[*] Rcvr1 Waiting for messages...
[*] Rcvr2 Waiting for messages...
[x] Rcvr1 Received 'Hello 0'
[x] Rcvr2 Received 'Hello 1'
[x] Rcvr1 Received 'Hello 2'
[x] Rcvr2 Received 'Hello 3'
[x] Rcvr1 Received 'Hello 4'
[x] Rcvr2 Received 'Hello 5'
[x] Rcvr1 Received 'Hello 6'
[x] Rcvr2 Received 'Hello 7'
[x] Rcvr1 Received 'Hello 8'
...
但是,首先啟動我的接收器會導致只有一個線程接收消息(要啟動的最后一個線程):
[*] Rcvr2 Waiting for messages...
[*] Rcvr1 Waiting for messages...
[x] Rcvr1 Received 'Hello 9'
[x] Rcvr1 Received 'Hello 10'
[x] Rcvr1 Received 'Hello 11'
[x] Rcvr1 Received 'Hello 12'
[x] Rcvr1 Received 'Hello 13'
[x] Rcvr1 Received 'Hello 14'
[x] Rcvr1 Received 'Hello 15'
...
有趣的是,如果我啟動發送器,然后啟動接收器,如上所述,然后再次啟動發送器(而接收器正在處理第一批)。 發送的第一條消息是串行處理的,而第二批消息是並行處理的,或者至少與剩余的線程一起處理。:
[*] Rcvr1 Waiting for messages...
[*] Rcvr2 Waiting for messages...
[x] Rcvr1 Received '[Batch 1] Hello 0'
[x] Rcvr1 Received '[Batch 1] Hello 1'
[x] Rcvr1 Received '[Batch 1] Hello 2'
[x] Rcvr1 Received '[Batch 1] Hello 3'
[x] Rcvr1 Received '[Batch 1] Hello 4'
[x] Rcvr1 Received '[Batch 1] Hello 5'
[x] Rcvr1 Received '[Batch 1] Hello 6'
[x] Rcvr1 Received '[Batch 1] Hello 7'
[x] Rcvr1 Received '[Batch 1] Hello 8'
[x] Rcvr2 Received '[Batch 2] Hello 1'
[x] Rcvr1 Received '[Batch 1] Hello 9'
[x] Rcvr2 Received '[Batch 2] Hello 3'
[x] Rcvr1 Received '[Batch 1] Hello 10'
[x] Rcvr2 Received '[Batch 2] Hello 5'
[x] Rcvr1 Received '[Batch 1] Hello 11'
[x] Rcvr2 Received '[Batch 2] Hello 7'
[x] Rcvr1 Received '[Batch 1] Hello 12'
[x] Rcvr2 Received '[Batch 2] Hello 9'
[x] Rcvr1 Received '[Batch 1] Hello 13'
[x] Rcvr2 Received '[Batch 2] Hello 11'
RabbitMQ 顯然可以做到這一點,我不確定我做錯了什么。 我的簡單代碼如下:
發件人
public class Send {
private final static String QUEUE_NAME = "hello";
public static void main(String[] argv) throws Exception {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
try (Connection connection = factory.newConnection();
Channel channel = connection.createChannel()) {
channel.queueDeclare(QUEUE_NAME, true, false, false, null);
for(int x=0; x<100; x++) {
String message = "Hello "+x;
channel.basicPublish("", QUEUE_NAME, null, message.getBytes("UTF-8"));
System.out.println(" [x] Sent '" + message + "'");
}
}
}
}
接收者
package com.mawv.ingest.rabbitmq;
import com.rabbitmq.client.*;
import java.util.concurrent.Executors;
import java.util.concurrent.ThreadPoolExecutor;
public class Recv {
private final static String QUEUE_NAME = "hello";
public static void main(String[] argv) throws Exception {
ThreadPoolExecutor rcvrPool = (ThreadPoolExecutor) Executors.newFixedThreadPool(10);
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Runnable rcvr1 = () -> {
try {
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, true, false, false, null);
System.out.println(" [*] Rcvr1 Waiting for messages...");
DeliverCallback deliverCallback = (consumerTag, delivery) -> {
Envelope envelope = delivery.getEnvelope();
String message = new String(delivery.getBody(), "UTF-8");
System.out.println(" [x] Rcvr1 Received '" + message + "'");
long deliveryTag = envelope.getDeliveryTag();
channel.basicAck(deliveryTag, true);
try {
Thread.sleep(1000);
} catch (Exception ex) { }
};
channel.basicConsume(QUEUE_NAME, false, deliverCallback, consumerTag -> { });
} catch(Exception ex){
ex.printStackTrace();
}
};
Runnable rcvr2 = () -> {
try {
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, true, false, false, null);
System.out.println(" [*] Rcvr2 Waiting for messages...");
DeliverCallback deliverCallback = (consumerTag, delivery) -> {
Envelope envelope = delivery.getEnvelope();
String message = new String(delivery.getBody(), "UTF-8");
System.out.println(" [x] Rcvr2 Received '" + message + "'");
long deliveryTag = envelope.getDeliveryTag();
channel.basicAck(deliveryTag, true);
try {
Thread.sleep(1000);
} catch (Exception ex) {
}
};
channel.basicConsume(QUEUE_NAME, false, deliverCallback, consumerTag -> {
});
} catch(Exception ex){
ex.printStackTrace();
}
};
rcvrPool.execute(rcvr1);
rcvrPool.execute(rcvr2);
}
}
我也綁定了這個例子,正如他們描述的那樣,看到了相同的結果。 https://self-learning-java-tutorial.blogspot.com/2015/09/rabbitmq-one-producer-and-multiple.html
我假設我的設置有問題。
根據RabbitMQ api:
“雖然一個 Channel 可以被多個線程使用,但確保一次只有一個線程執行一個命令很重要。同時執行命令可能會導致拋出 UnexpectedFrameError”
首先,我認為您應該為不同的線程使用不同的通道。
最后,我認為第一個線程因為空閑而終止,所以只有第二個線程處於活動狀態並完成整個工作。 在這種情況下,一根線程就足夠了。
看看 java 8 的 ThreadPoolExecutor api:
https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ThreadPoolExecutor.html
例如,您可以找到:
“默認情況下,即使是核心線程也只有在新任務到達時才最初創建和啟動,但這可以使用方法 prestartCoreThread() 或 prestartAllCoreThreads() 動態覆蓋。如果您使用非空構造池,您可能想要預啟動線程隊列”
和
“如果池中當前有超過 corePoolSize 的線程,如果空閑時間超過 keepAliveTime(請參閱 getKeepAliveTime(TimeUnit)),多余的線程將被終止。”
您應該使用 prestartAllCoreThreads() 或 prestartCoreThreads() 使核心線程即使在空閑時也能啟動,或者使用 getKeepAliveTime(TimeUnit) 使它們即使空閑也能保持活動狀態。
看起來我缺少一個關鍵的通道配置。 這解決了我的問題:
channel.basicQos(1);
這就是 RabbitMQ 所說的。
公平調度
您可能已經注意到調度仍然不能完全按照我們想要的方式工作。 例如,在有兩個 worker 的情況下,當所有奇數消息都很重,偶數消息很輕時,一個 worker 會一直很忙,而另一個 worker 幾乎不做任何工作。 好吧,RabbitMQ 對此一無所知,仍然會均勻地發送消息。
發生這種情況是因為 RabbitMQ 只是在消息進入隊列時分派消息。 它不考慮消費者未確認消息的數量。 它只是盲目地將每條第 n 條消息分派給第 n 條消費者。
為了解決這個問題,我們可以使用帶有 prefetchCount = 1 設置的 basicQos 方法。 這告訴 RabbitMQ 一次不要給一個工人多個消息。 或者,換句話說,在處理並確認前一條消息之前,不要向工作人員發送新消息。 相反,它會將它分派給下一個不忙的工人。
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.