Reading a topic of kafka with react

Question

I'll start by saying I'm sorry if this question looks generic, but I'm having trouble to solve this. So I'll give it a shot here.

I want to build a consumer of a kafka topic with react, so it'll render I don't know something like a grid everytime there's a new message in my topic.

I already have the code of the consumer:

var kafka = require('kafka-node'),
Consumer = kafka.Consumer,
client = new kafka.Client(),
consumer = new Consumer(
    client,
    [
        { topic: 'logs', partition: 0 }
    ],
    {
        autoCommit: false
    }
),
Producer = kafka.Producer,
client = new kafka.Client(),
producer = new Producer(client);

consumer.on('message', function (message) {
    console.log(message);
});

anytime there's a new message the consumer's event "on" will be called.

But I can't see a way to hook it up with react.

I'm up for anything, a tutorial, article, anything at all.

Thanks.

Answer 1

Here's an example of what you could do. In essence, have the component that will render the message information observe the kafka consumer and set that message in the component's state. The component will then render with the new message in state (any time setState is called, causes the component to update itself).

This assumes you want to display a list of messages and the consumer.on callback provides a single message that needs to be added to the list.

var MessageDetails = React.createClass({
    //create a render function for this to render the details about a message
});

var MessageDisplay = React.createClass({
    getInitialState: function() {
        return { messages: [] };
    },

    onComponentDidMount: function() {
        consumer.on('message', function (message) {
            // This updates the state so component will re-render
            var messageList = this.state.messages;
            messageList.push(message);
            this.setState({messages: messageList});
        }.bind(this));
    },

    onComponentWillUnmount: function() {
        // Unsubscribe from the consume here.
    },

    render: function() {
        var messageList = this.state.messages.map(function(message) {
            return (<MessageDetails id={message.id} etc. />);
        });
        return (
          <div className="commentBox">
              {messageList}
          </div>
        );
    }
});

Answer 2

You should use websocket Kafka proxy like https://github.com/Effyis/kafka2websocket or https://github.com/Microsoft/kafka-proxy-ws , because, AFAIK, there are no browser compatible clients available yet. After that, you will be able to interact with Kafka through websockets

Answer 3

You can use lightstreamer for the same purpose, you can find more information here https://lightstreamer.com/doc.htm .