在队列中找不到MSMQ有毒消息

Question

Have encounted a strange issue with poisoned messages in an MSMQ queue. When a poisoned message is detected I'm using the code below to handle the exception and move the message to the poison queue, but this fails because the message is not found even though I get its lookupId from the thrown exception. See relevant code below.

public bool HandleError(Exception error)
{
    var poisonException = error as MsmqPoisonMessageException;
    if (null == poisonException) return false;

    var lookupId = poisonException.MessageLookupId;

    var queuePath = Environment.MachineName + "\\" + ConfigurationManager.AppSettings["QueuePath"];
    var poisonQueuePath = Environment.MachineName + "\\" + ConfigurationManager.AppSettings["PoisonQueuePath"];

    var orderQueue = new System.Messaging.MessageQueue(queuePath);
    var poisonMessageQueue = new System.Messaging.MessageQueue(poisonQueuePath);

    // Use a new transaction scope to remove the message from the main queue and add it to the poison queue.
    using (var txScope = new TransactionScope(TransactionScopeOption.RequiresNew))
    {
        int retryCount = 0;
        while (retryCount < 3)
        {
            retryCount++;

            try
            {
                // Try to get the poison message using the look up id. This line throws InvalidOperationException
                var message = orderQueue.ReceiveByLookupId(lookupId);
                // Send the message to the poison message queue.
                poisonMessageQueue.Send(message, System.Messaging.MessageQueueTransactionType.Automatic);

                txScope.Complete();

                Logger.Debug("Moved poisoned message with look up id: " + lookupId + " to poison queue: " + ConfigurationManager.AppSettings["PoisonQueuePath"]);
                break;
            }
            catch (InvalidOperationException e)
            {
                if (retryCount < 3)
                {
                    Logger.Debug("Trying to move message to poison queue but message is not available, sleeping for 10 seconds before retrying", e);
                    Thread.Sleep(TimeSpan.FromSeconds(10));
                }
                else
                {
                    Logger.Debug("Giving up on trying to move the message", e);
                }
            }
        }
    }

    Logger.Info("Restarting the service to process rest of the messages in the queue");
    WaitCallback restartCallback = new WaitCallback(Start);
    ThreadPool.QueueUserWorkItem(restartCallback);

    return true;
}

This code is basically copied from Microsoft's example code here .

The error thrown is of the correct type:

System.ServiceModel.MsmqPoisonMessageException: The transport channel detected a poison message.

But when attempting to get the message from the queue I get:

System.InvalidOperationException: Message requested was not found in the queue specified.

My first thought was that the queues might not have the correct permissions set but I've double checked that the Network Service user has all the necessary rights to read and write messages to both queues.

It's worth mentioning that this code has been working perfectly in production for months, and has survived many poisoned messages in the past. Any input on what might have caused this issue is greatly appreciated.

Answer 1

This will happen when you have more than one retry cycle specified. If your maxRetryCycles is greater than zero and your retryCycleDelay is greater than 30 seconds, you will see the problem you describe. The message is actually sitting in a subqueue called "retry" as it waits the retryCycleDelay between cycles. So when your IErrorHandler looks for it in the "main" queue, it won't find it. For some reason, WCF throws the MsmqPoisonMessageException at the end of each retry cycle, NOT just once at the end of all the retry cycles. Which means that your IErrorHandler will get called at the end of each cycle. Seems really strange to me but that's the way it is.

A better approach now days (if you can guarantee that your code will have MSMQ 4.0) is to change your receiveErrorHandling from "Fault" to "Move" and then get rid of your IErrorHandler. With that approach the messages will be moved for you after all the retries and retry cycles have completed. It is moved to a subqueue called "poison".

See here for more details:

Poison Message Handling in MSMQ 4.0