Service Bus how to reconnect after servers restart (non-azure)

Question

I'm using Service Bus 1.1 on Premise, and I'm building a WindowsService which should just be listening on a specific queue, running on another machine.

The code of the WindowsService to connect to SB is like this:

MessagingFactory _receiverFactory = MessagingFactory.CreateFromConnectionString(sbConnectionString);
MessageReceiver _messageReceiver = _receiverFactory.CreateMessageReceiver(listeningQueue);

BrokeredMessage brokeredMessage = _messageReceiver.Receive(TimeSpan.FromSeconds(60));

Everything works fine until I restart the servers: I noticed that if the WindowsService is starting BEFORE all the services of Service Bus are available, my application will never connect at all, until I manually restart my WindowsService.

Actually, if the SB is not fully running at the time I'm building the MessagingFactory, the Receive instruction will always fail, even if the SB services are coming back in a second time...

I've tried to intercept the exceptions (MessagingCommunicationException) and to rebuild the MessagingFactory, but is simply not working, the only way to reconnect is to restart the WindowsService.

Any suggestions?

Answer 1

1. Setup a High Availability ("HA") Service Farm.

https://blogs.technet.microsoft.com/meamcs/2013/12/08/recommended-practices-service-bus-for-windows-server/

http://www.planetgeek.ch/2014/12/10/service-bus-for-windows-server-high-availability/

But even if you do or do not do #1 "HA", you should do #2.

2. Use " Polly " to catch and auto-retry SPECIFIC exceptions

    using System; using System.Collections.Generic; using System.Linq; using System.Threading; using Microsoft.ServiceBus; using Microsoft.ServiceBus.Messaging; using Polly; private static void KeepReadingTheQueue(QueueClient qc) { int retries = 0; int eventualFailures = 0; Policy pol = Policy.Handle<System.OperationCanceledException>(ex => null != ex) .Or<UnauthorizedAccessException>(ex => null != ex) .Or<Microsoft.ServiceBus.Messaging.MessagingCommunicationException>() .Or<Microsoft.ServiceBus.Messaging.MessageLockLostException>() .WaitAndRetryForever( sleepDurationProvider: attempt => TimeSpan.FromMilliseconds(500), // Wait 500ms between each try. onRetry: (exception, calculatedWaitDuration, context) => // Capture some info for logging! { ReportDuplicatesOrMissing("Policy.OnRetry", batchNumbers); // This is your new exception handler! // Tell the user what they've won! Console.WriteLine("Log, then retry: " + exception.Message, ConsoleColor.Yellow); retries++; }); int i = 0; int receiveBatchSize = 1; /* i have this in a custom settings class, hard coded here */ while (true) { i++; ReportDuplicatesOrMissing("while.true", batchNumbers); try { // Retry the following call according to the policy - 15 times. pol.Execute(() => { IEnumerable<BrokeredMessage> messages = null; if (receiveBatchSize <= 1) { // Receive the message from the queue BrokeredMessage receivedMessage = qc.Receive(); List<BrokeredMessage> listMsgs = new List<BrokeredMessage>(); listMsgs.Add(receivedMessage); messages = listMsgs as IEnumerable<BrokeredMessage>; } else { messages = qc.ReceiveBatch(receiveBatchSize); int count = messages.Count(); if (count > 0) { Console.WriteLine("ReceiveBatch.Count='{0}'", count); } } foreach (BrokeredMessage receivedMessage in messages) { if (receivedMessage != null) { MyPayLoadObject concreteMsgObject = receivedMessage.GetBody<MyPayLoadObject>(); receivedMessage.Complete(); Console.WriteLine(@"Message received: {0}", concreteMsgObject); } else { Console.WriteLine(@"No new messages in the queue"); Thread.Sleep(1000); } Thread.Sleep(100); } }); } catch (Exception e) { Console.WriteLine("Request " + i + " eventually failed with: " + e.Message, ConsoleColor.Red); eventualFailures++; } } } 

packages.config

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="Microsoft.WindowsAzure.ConfigurationManager" version="2.0.1.0" targetFramework="net45" />
  <package id="Polly" version="4.3.0" targetFramework="net45" />
  <package id="WindowsAzure.ServiceBus" version="2.1.4.0" targetFramework="net45" />
</packages>