Message Dispatcher with Azure Service Bus

Question

I'm trying to code up a Message Dispatcher pattern using Azure Service Bus. The basic idea is that I have a number of clients which maintain TCP connections with a central server.

What I'd like to be able to do is to have messages posted on a Service Bus Topic, that would then be consumed by this central server, and then dispatched to the corresponding client. I think the correct way to approach this problem is to have the semantic equivalent of one queue per client, which would be useful especially since clients can disconnect at will, and the messages should be held on the queue until they reconnect. I'd like to have strong guarantees for message delivery, so I think Peek-Lock would work best here (and not Receive and Delete).

I can think of a few ways of doing this:

1. Maintaining a separate queue per client. I think this is too heavyweight and naive of a solution.

2. Initiating n subscriptions (where n is the number of clients) on the central server, and when a new message comes in, dispatching it to the appropriate client. This would also involve detecting when the clients have disconnected or cannot be reached, and not checking those subscriptions until they've reconnected. This can work, and with heavy use of Task and Async should be able to work fairly well, but it also seems wasteful.

3. The client server only has one subscription to the topic, but has a SqlFilter which only "listens" to messages for clients that are currently connected. This would involve changing the filter every time a client connects/disconnects, and detecting that a message is undeliverable because a client just disconnected and leaving that message on the queue.

I think the last solution is probably the best one in terms of scalability: I'm still in the process of coding these examples up, so can't be sure until I test. Anyone have any guidance on this?

Answer 1

the issue with the 3rd solution from above is that if you are going to update the filter every time a client disconnects, then any new messages that are meant to the client when the client is disconnected will be dropped by the topic.. based on your description of the problem, you don't want that.

Hence since you want to be able to retain the messages even when the client is disconnected, you would have to go with a single subscription per client. (2 above).. If you do have the case where some clients might just vanish and never come back, then you can use the topic feature which automatically deletes subscriptions that are idle.. where the SERVICEBUS topic will garbage collect unused subscriptions.

Note that the one gotcha with 2 is that we only support 2000 subscriptions on a given topic.

There is no downside to the 2nd option.. if the client has disconnected you can stop receiving from the subscription.

You should however make sure that a subscription that you are not receiving from is not growing too much in size.. Because you don't want a single subscription to eat up the entire space of the topic just because you have stopped consuming messages from it