Listener for NATS JetStream

Question

Can some one help how to configure NATS jet stream subscription in spring boot asynchronously example: looking for an equivalent annotation like @kafkalistener for Nats jetstream

I am able to pull the messages using endpoint but however when tried to pull messages using pushSubscription dispatcherhandler is not invoked. Need to know how to make the listener to be active and consume messages immediately once the messages are published to the subject.

Any insights /examples regarding this will be helpful, thanks in advance.

Answer 1

I don't know what is your JetStream retention policy, neither the way you want to subscribe. But I have sample code for WorkQueuePolicy push subscription, wish this will help you.

public static void subscribe(String streamName, String subjectKey,
                             String queueName, IMessageHandler iMessageHandler) throws IOException,
        InterruptedException, JetStreamApiException {
    long s = System.currentTimeMillis();
    Connection nc = Nats.connect(options);
    long e = System.currentTimeMillis();
    logger.info("Nats Connect in " + (e - s) + " ms");
    JetStream js = nc.jetStream();
    Dispatcher disp = nc.createDispatcher();
    MessageHandler handler = (msg) -> {
        try {
            iMessageHandler.onMessageReceived(msg);
        } catch (Exception exc) {
            msg.nak();
        }
    };
    ConsumerConfiguration cc = ConsumerConfiguration.builder()
            .durable(queueName)
            .deliverGroup(queueName)
            .maxDeliver(3)
            .ackWait(Duration.ofMinutes(2))
            .build();
    PushSubscribeOptions so = PushSubscribeOptions.builder()
            .stream(streamName)
            .configuration(cc)
            .build();
    js.subscribe(subjectKey, disp, handler, false, so);
    System.out.println("NatsUtil: " + durableName + "subscribe");
}

IMessageHandler is my custom interface to handle nats.io received messages.

Answer 2

First, configure the NATS connection. Here you will specify all your connection details like server address(es), authentication options, connection-level callbacks etc.

Connection natsConnection = Nats.connect(
            new Options.Builder()
                    .server("nats://localhost:4222")
                    .connectionListener((connection, eventType) -> {})
                    .errorListener(new ErrorListener(){})
                    .build());

Then construct a JetStream instance

JetStream jetStream = natsConnection.jetStream();

Now you can subscribe to subjects. Note that JetStream consumers can be durable or ephemeral, can work according to push or pull logic. Please refer to NATS documentation ( https://docs.nats.io/nats-concepts/jetstream/consumers ) to make the appropriate choice for your specific use case. The following example constructs a durable push consumer:

    //Subscribe to a subject.
    String subject = "my-subject";

    //queues are analogous to Kafka consumer groups, i.e. consumers belonging
    //to the same queue (or, better to say, reading the same queue) will get
    //only one instance of each message from the corresponding subject
    //and only one of those consumers will be chosen to process the message
    String queueName = "my-queue";

    //Choosing delivery policy is analogous to setting the current offset
    //in a partition for a consumer or consumer group in Kafka.
    DeliverPolicy deliverPolicy = DeliverPolicy.New;
    PushSubscribeOptions subscribeOptions = ConsumerConfiguration.builder()
            .durable(queueName)
            .deliverGroup(queueName)
            .deliverPolicy(deliverPolicy)
            .buildPushSubscribeOptions();
    Subscription subscription = jetStream.subscribe(
            subject,
            queueName,
            natsConnection.createDispatcher(),
            natsMessage -> {
                //This callback will be called for incoming messages
                //asynchronously. Every subscription configured this
                //way will be backed by its own thread, that will be
                //used to call this callback.
            },
            true,  //true if you want received messages to be acknowledged
                   //automatically, otherwise you will have to call
                   //natsMessage.ack() manually in the above callback function
            subscribeOptions);

As for the declarative API (ie some form of @NatsListener annotation analogous to @KafkaListener from Spring for Apache Kafka project), there is none available out of the box in Spring. If you feel like you absolutely need it, you can write one yourself, if you are familiar with Spring BeanPostProcessor-s or other extension mechanism that can help to do that. Alternatively you can refer to 3rd party libs, it looks like a bunch of people (including myself) felt a bit uncomfortable when switching from Kafka to NATS, so they tried to bring the usual way of doing things with them from the Kafka world. Some examples can be found on github:

• https://github.com/linux-china/nats-spring-boot-starter ,
• https://github.com/dstrelec/nats
• https://github.com/amalnev/declarative-nats-listeners

There may be others.