Tracking messages in chat system

Question

we have build a chat system in Node.JS. Where we have three channels for delivering messages one using mqtt protocol second using third party service pusher channels and third message fetch service based on gcm received. Once the message is sent from one user to second user its stored in redis untill its delivered to second user. the problem we are facing is that we are unable to track the missing messages which are undelivered any idea how can we track the messages in chat?

We tried ack of messages from client side. But sometimes due to api failure etc we are unable to get acks too.Due to which we are unable to track the messages.

Also i researched about messaging systems some of them use queue based messaage broker. I am thinking to use rabbit mq for this purpose. Can someone explain weather message broker will bring more clarity to delivery of messages?

Answer 1

Maybe (unless an specific case) pub/sub may not be the right tool you need here the issue with pub/sub (mqtt) for this case is that this is a broadcast publishing, which means that you are just publishing a message for whoever is subscribed in the topic. Pub/Sub has something called Quality of Service (QoS) that has 3 levels where, the the behavior of the broker treating this message will be depending on the reliability of the delivery you want:

QoS are:

• At most once (0)
• At least once (1)
• Exactly once (2)

QoS level 2 may be the one you need, where the message will be retried until it's delivered to the receiver, so if it (receiver) is unavailable, broker will keep trying to deliver the message, but the issue with QoS lvl 2 is that is not available on all pub/sub services (I think that Redis doesn't have it, at least that I know)... and second, you need to know who are all the receivers in order to establish this QoS level.

Here's where you can find more details about QoS: https://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-quality-of-service-levels/

I think that the best (and maybe simplest) tools that you need for this project are

1. Document data store (NoSQL) for message storage with UUIDs and timestamps.
2. Web socket to notify all connected clients about new messages. This will be the channel to also deliver new incoming messages (and can also work to fetch for old messages, but depends on levels of demand).
3. Local storage to keep messages while chat server is unavailable.

And that's it.

Maybe it may look simpler to make use of a pub/sub service because it deals with the complexity of receiving, managing, and delivering messages, but they are more designed to be decoupled with clients that are coming in and going out over time without been aware of others.