如何在 Spring Boot 中使用阻塞队列？

Question

I am trying to use BlockingQueue inside Spring Boot. My design was like this: user submit request via a controller and controller in turn puts some objects onto a blocking queue. After that the consumer should be able to take the objects and process further.

I have used Asnyc, ThreadPool and EventListener. However with my code below I found consumer class is not consuming objects. Could you please help point out how to improve?

Queue Configuration

@Bean
public BlockingQueue<MyObject> myQueue() {
    return new PriorityBlockingQueue<>();
}

@Bean
public Executor getAsyncExecutor() {
    ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
    executor.setCorePoolSize(3);
    executor.setMaxPoolSize(3);
    executor.setQueueCapacity(10);
    executor.setThreadNamePrefix("Test-");
    executor.initialize();
    return executor;
}

Rest Controller

@Autowired
BlockingQueue<MyObject> myQueue;

@RequestMapping(path = "/api/produce")
public void produce() {
    /* Do something */
    MyObject myObject = new MyObject();
    myQueue.put(myObject);
}

Consumer Class

@Autowired
private BlockingQueue<MyObject> myQueue;

@EventListener
public void onApplicationEvent(ContextRefreshedEvent event) {
    consume();
}

@Async
public void consume() {
    while (true) {
        try {
            MyObject myObject = myQueue.take();
        }
        catch (Exception e) {
        }
    }
}

Answer 1

Your idea is using Queue to store messages, consumer listens to spring events and consume. I didn't see your code have actually publish the event, just store them in queue . If you want to use Spring Events, producers could like this:

@Autowired
    private ApplicationEventPublisher applicationEventPublisher;

    public void doStuffAndPublishAnEvent(final String message) {
        System.out.println("Publishing custom event. ");
        CustomSpringEvent customSpringEvent = new CustomSpringEvent(this, message);
        applicationEventPublisher.publishEvent(customSpringEvent);
    }

check this doc

If you still want to use BlockingQueue , your consumer should be a running thread, continuously waiting for tasks in the queue, like:

public class NumbersConsumer implements Runnable {
    private BlockingQueue<Integer> queue;
    private final int poisonPill;

    public NumbersConsumer(BlockingQueue<Integer> queue, int poisonPill) {
        this.queue = queue;
        this.poisonPill = poisonPill;
    }
    public void run() {
        try {
            while (true) {
                Integer number = queue.take(); // always waiting 
                if (number.equals(poisonPill)) {
                    return;
                }
                System.out.println(Thread.currentThread().getName() + " result: " + number);
            }
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
    }
}

could check this code example

Answer 2

In the end I came up with this solution.

Rest Controller

@Autowired
BlockingQueue<MyObject> myQueue;

@RequestMapping(path = "/api/produce")
public void produce() {
    /* Do something */
    MyObject myObject = new MyObject();
    myQueue.put(myObject);
    Consumer.consume();
}

It is a little bit weird because you have to first put the object on queue yourself then consume that object by yourself. Any suggestions on improvement is highly appreciated.

Answer 3

@Async doesn't actually start a new thread if the target method is called from within the same object instance, this could be the problem in your case. Also note that you need to put @EnableAsync on a config class to enable the @Async annotation.

See Spring documentation: https://docs.spring.io/spring-framework/docs/current/reference/html/integration.html#scheduling-annotation-support

The default advice mode for processing @Async annotations is proxy which allows for interception of calls through the proxy only. Local calls within the same class cannot get intercepted that way. For a more advanced mode of interception, consider switching to aspectj mode in combination with compile-time or load-time weaving.