Spring Kafka Listener on Http Request

Question

I am using Spring and Kafka, I make an HTTP POST request as shown below and send some info to another service via a Kafka topic.

@RequestMapping(method = RequestMethod.POST, value = "/portfolio")
public void getPortfolio(
       Authentication auth,
    @RequestBody User user
) {
    //Data Transfer Object
    UserDTO dto = user.toDTO();
    dto.setId(((AuthenticatedUser) auth.getPrincipal()).getId());

    //Sending message to Kafka topic
    sender.sendPortfolioRequest(dto);
}

I then want to listen for the response on a different topic and return data in the HTTP response but am stuck here. I am able to listen to for the response using the below listener method but don't know how to put the two together.

@KafkaListener(
    topics = Topics.PORTFOLIO_RESULT,
    containerFactory = "portfolioKafkaListenerContainerFactory"
)
public void portfolioListener(UserPortfolioDTO portfolio) {
    System.out.println("Recieved Portfolio: " + portfolio.toString());
}

PS I am new to using HTTP requests and don't know if this is the correct way to do what I'm trying to achieve or if I should be creating a new resource with the POST and redirecting to that or something else.

Answer 1

This can't be done with the @KafkaListener since it is started separately and fully works in its own thread(s). Meanwhile you expect a reply in the HTTP request thread.

Only the solution possible for you here is a ConsumerFactory and manual usage of the Apache Kafka Consumer . So, you send, you obtain a Consumer instance from the factory, call its poll() being blocked until a result, build a response for HTTP and close Consumer .

Answer 2

According to what I'm understanding, I suggest you to enable asych http request to be able to link your process.

Creating Asynchronous Methods with springboot

This option will allow you to process sendPortfolioRequest and free the http request (otherwise you will have an http request timeout from client side).

And what you try to do looks like an anti-pattern : you want to do link a sychron http request (because your http client is waiting a response from the server) and an asynchron messaging system (Kafka).

To be able to do what you want, I suggest you to change your http endpoint and add a websocket to be in the best practice.

Please see Using WebSocket to build an interactive web application