Thread aborted exceptions in wcf service

Question

I have a WCF service (built in .NET framework 3.5) hosted on IIS 6.0.

The flow of the code is as follows

1. Client (which is another web service) calls the WCF service
2. WCF services invokes a thread to do the processing in background and responds to the callee immediately.
3. The background thread after completing all processing, calls back the thread. This call is basically an HTTPs request as the client is a web service.

I am load testing my WCF service to define the thresholds. The observation is as follows:

Around 3 iterations of 1024 requests made to WCF service within 1 minute pass successfully. Time taken to complete each iteration is around 25-30 mins. However from 4th iteration bulk failures are seen. Around 50% of the requests fail with the below exception.

Exception-Thread was being aborted.

Stack trace

21_10_2016_09_30_52,9:30:52 AM,Information,Thread name- apSwTTbLTETfwT3y Stack trace in ProcessTestConversion method -    at System.Threading.WaitHandle.WaitOneNative(SafeHandle waitableSafeHandle, UInt32 millisecondsTimeout, Boolean hasThreadAffinity, Boolean exitContext)
   at System.Threading.WaitHandle.InternalWaitOne(SafeHandle waitableSafeHandle, Int64 millisecondsTimeout, Boolean hasThreadAffinity, Boolean exitContext)
   at System.Threading.WaitHandle.WaitOne(Int32 millisecondsTimeout, Boolean exitContext)
   at System.Net.LazyAsyncResult.WaitForCompletion(Boolean snap)
   at System.Net.Connection.SubmitRequest(HttpWebRequest request, Boolean forcedsubmit)
   at System.Net.ServicePoint.SubmitRequest(HttpWebRequest request, String connName)
   at System.Net.HttpWebRequest.SubmitRequest(ServicePoint servicePoint)
   at System.Net.HttpWebRequest.GetRequestStream(TransportContext& context)
   at System.Net.HttpWebRequest.GetRequestStream()
   .
   .(My function calls stack trace)
   .
   .

The changes I tried to solve this problems are as follows:

   <behavior>
    <serviceThrottling maxConcurrentCalls="2000"
                                 maxConcurrentInstances ="2400"
                                 maxConcurrentSessions ="400"/>
    </behavior>

in web.config

<system.web>
        <compilation debug="false" />
        <httpRuntime executionTimeout="1800"/>
    </system.web>

in web.config

  <system.net>
        <connectionManagement>
          <add address = "*" maxconnection = "100" />
        </connectionManagement>
      </system.net>

in web.config

ServicePointManager.DefaultConnectionLimit = 100; (Change in code)

I have set the IdleTimeout property of the App pool to 0 as suggested by many people on StackOverflow.

Where ever the streams are used I have dispose at all the places. So all the streams are closed.

Can any one tell me who is aborting the threads and why and are there any means or tool to trace the cause for thread abort initiation?

Answer 1

I have come across this issue and it was down to an improper use of the client class.

What was happening is when a client class has been instantiated it would not release the resources back causing a reduction in throughput. A very unhelpful exception "Thread was being aborted" would occur. This was resolved by creating a helper class that generic ally created a client object and then implemented the constructor and dispose method correctly.

Some IIS exceptions aren't very helpful or true to the actual cause of the problem but to get to bottom of what was done to resolve my issue was to look at the IIS logs. Specifically " Failed Request Tracing Rules "

Hopefully this will help, I can understand your frustration it was a headache to resolve.

Answer 2

I have same faced thing before. Solution was to free up resources/client after its scope has been ended by implementing Dispose method.

Problem exists since the C# using statement to automatically clean up resources when using a typed client is not successful and any exception thrown is masked by implicit dispose which eats actual exception & throws some other exception like timeout or other exceptions.

The C# "using" statement results in a call to Dispose(). This is the same as Close(), which may throw exceptions when a network error occurs. Because the call to Dispose() happens implicitly at the closing brace of the "using" block, this source of exceptions is likely to go unnoticed both by people writing the code and reading the code. This represents a potential source of application errors. (from MSDN )

You should free up resources like :

try
{
    ...
    client.Close();
}
catch (CommunicationException e)
{
    ...
    client.Abort();
}
catch (TimeoutException e)
{
    ...
    client.Abort();
}
catch (Exception e)
{
    ...
    client.Abort();
    throw;
}

In this way you can not only find actual source of exception & also free up resource when some execption occurs.

Other alternative solution can be implementing Dispose like :

/// <summary>
/// Calculator Client
/// </summary>
public partial class CalculatorClient : IDisposable
{
    #region IDisposable implementation

    /// <summary>
    /// IDisposable.Dispose implementation, calls Dispose(true).
    /// </summary>
    void IDisposable.Dispose()
    {
        Dispose(true);
    }

    /// <summary>
    /// Dispose worker method. Handles graceful shutdown of the
    /// client even if it is an faulted state.
    /// </summary>
    /// <param name="disposing">Are we disposing (alternative
    /// is to be finalizing)</param>
    protected virtual void Dispose(bool disposing)
    {
        if (disposing)
        {
            try
            {
                if (State != CommunicationState.Faulted)
                {
                    Close();
                }
            }
            finally
            {
                if (State != CommunicationState.Closed)
                {
                    Abort();
                }
            }
        }
    }

    /// <summary>
    /// Finalizer.
    /// </summary>
    CalculatorClient()
    {
        Dispose(false);
    }

    #endregion
}

Sources :

Avoiding Problems with the Using Statement MSDN

Using and Disposing of WCF Clients