创建许多消费者的RabbitMQ最佳实践

Question

We are just starting to use RabbitMQ with C#. My current plan is to configure in the database the number and kind of consumers to run on a given server. We have an existing windows service and when that starts I want to spawn all of the RabbitMQ consumers. My question is what is the best way to spwan these from a windows service?

My current plan is to read the configuration out of the database and spawn a long running task for each consumer.

                var t = new Task(() =>
                {
                    var instance = LoadConsumerClass(consumerEnum, consumerName);
                    instance.StartConsuming();//blocking call
                }, TaskCreationOptions.LongRunning);
                t.Start();

Is this better or worse than creating a thread for each consumer?

                var messageConsumer = LoadConsumerClass(consumerEnum, consumerName);
                var thread = new Thread(messageConsumer.StartConsuming);

I'm hoping that more than a few others have already tried what I'm doing and can provide me with some ideas for what worked well and what didn't.

Answer 1

In EasyNetQ we have a single dispatcher thread for all consumers on a single connection. We also provide a facility to to return a Task from the message handler, so it's easy to do async IO if you want to make a database call, go to the file system, or make a web service request.

Having said that it's perfectly legitimate to have each consumer consuming on a different thread. I guess it depends on your message throughput, how many consumers you have and the nature of your message handlers.

Answer 2

I'd stick with Tasks as they give you more features and generally allow for less boilerplate code. And, If I understand your code correctly, you'd be sharing a channel (IModel) in second case. This might cause troubles as the default IModel implementation is not thread safe (or used to be). There're more subtle nuances regarding thread safety you'd have to watch out. But it depends on your usage patterns. If you don't expect many messages/sec on each consumer, or if your app can handle messages fast then perhaps a single thread for all consumers will be you best option.

Answer 3

Task is great, but you not really going to use all the stuff it can do. The only thing you need is to do work in parallel. I faced the same question couple of months ago, what I finished with - is a thread per computation type (per queue) which is blocking on message arrival and doesn't consume cpu when waiting for messages. Open a new channel for each one of the threads. As for connections - if you application is meant to deal with high load of messages, I suggest you opening connection for every X workers (figure you your X), since only one channel can send the messages through the connection, so assuming one worker is consuming large message the others are blocked on connection level waiting it to be free.