Using Apache Kafka for real time messaging

Question

We are having an online examination system already developed in Spring MVC along with live webcam image capturing and uploading for offline proctoring, now clients are demanding for online or live proctoring solution where an online proctor will see the live images of an online candidate and based on that (if any malpractice he/she feels) can send warning messages to the online candidate.

To send warning messages from online proctor to candidate we are thinking to use Apache kafka, so message from online proctoring server (Spring Boot) will go to kakfa server and each exam server (Spring MVC) will consume the message. We are clear till this point but after receiving the message at exam server how to show that message to a particular candidate without refreshing or doing REST call from browser to exam server? or is there any other elegant way to do this?

Answer 1

If you need to have messages popup immediately on user's screen, you will probably want to go with setting up WebSockets. With WebSockets the candidate's browser will establish and keep a connection to the Exam server, and the exam server will be able to push messages to the browser.

This article explains it quite well: https://medium.com/swlh/websockets-with-spring-part-1-http-and-websocket-36c69df1c2ee

See also https://www.baeldung.com/websockets-spring

Answer 2

As you've clearly identified in your diagram, exposing Kafka to the Internet directly is not the way to go, and instead you need an intermediary to distribute the message data. This is typically a message broker or ideally an realtime messaging service designed specifically for Internet delivery of data. See a simplified diagram below.

To keep your stack simple, secure and scalable, I strongly recommend you consider an intermediary internet-facing message broker in the middle between your clients and Kafka.

This entire topic is covered in huge depth in " How to stream Kafka messages to Internet-facing clients over WebSockets "