兔子 mq 预取不明白

Question

I understand below

prefetch simply controls how many messsages the broker allows to be outstanding at the consumer at a time. When set to 1, this means the broker will send 1 message, wait for the ack, then send the next.

but questions regarding following scenarios:

1. Lets say prefetch is 200, we have 2 consumers idle. Broker got 150 messages, I think broker will pick one random and will send all 150 messages? I think yes it wont do sharing between consumers.

2. Lets say one consumer is having 100 messages in unack and one is idle and prefetch again is 200. Now we got 50 more messages, again I think broker will give those 50 to either one randomly? Or it will not give to consumer who already have 100 messages that not acked yet

3. If prefetch is 200, one consumer got 200, will listener block that thread (spring rabbitmq listner method) to send ack until all 200 processed? I think it will not send ack one by one and will wait until all prefetched messages processed. In other words if prefetch is 200 and if broker delivers 200 messages, when broker will start getting ack?

Answer 1

If there are two active consumers, the broker will distribute new messages fairly (until each instance has 200 outstanding).

If there are 150 messages in the queue and no consumers running; the first consumer to start will (likely) get all 150, but when both are running, the distribution is fair.

If there are 200 outstanding at each consumer, the broker will send new messages on demand as each one is ack'd. The consumer thread is not "blocked", it is just that the broker will send no more messages.

By default, spring will ack each message one-at-a-time. This behavior can be changed by setting the container's batchSize property. eg if it is set to 100, it will send an ack every 100 records; this improves performance, but adds the risk of duplicate deliveries after a failure. In this case, the broker will send up to 100 new messages after the ack.

In older versions, batchSize was called txSize .

EDIT

See this for an example; the default prefetch is 250 in recent versions.

@SpringBootApplication
public class So65201334Application {

    public static void main(String[] args) {
        SpringApplication.run(So65201334Application.class, args);
    }

    @RabbitListener(id = "foo", queues = "foo", autoStartup = "false")
    @RabbitListener(id = "bar", queues = "foo", autoStartup = "false")
    void listen(String in, @Header(AmqpHeaders.CONSUMER_TAG) String tag) throws InterruptedException {
        System.out.println(tag);
        Thread.sleep(240_000);
    }

    @Bean
    public ApplicationRunner runner(RabbitTemplate template, RabbitListenerEndpointRegistry registry) {
        return args -> {
            for (int i = 0; i < 200; i++) {
                template.convertAndSend("foo", "bar");
            }
            registry.getListenerContainer("foo").start();
            System.out.println("Hit Enter to start the second listener and send more records");
            System.in.read();
            registry.getListenerContainer("bar").start();
            Thread.sleep(2000);
            for (int i = 0; i < 200; i++) {
                template.convertAndSend("foo", "bar");
            }
        };
    }

}

As expected, all 200 went to the first consumer:

When the second consumer is started, the records are sent to both consumers, not the one that has no backlog. With the distribution now looking like this:

When I increase the prefetch to 400, you can see that the new messages go 50% to each consumer.