IIS应用程序池意外崩溃

Question

We are facing a very tricky issue for almost one week now and we are still unable to solve it.

The issue is that we have an application pool on IIS who is crashing and recycling several times a day.

We monitored what's happening on event viewer but the error there was not enough detailed.

Example of the 2 errors (2 seconds between them) from windows event viewer :

Error 1 :

Event Id : 1000

An unhandled exception occurred and the process was terminated.

Application ID: /LM/W3SVC/20/ROOT

Process ID: 14300

Exception: System.ObjectDisposedException

Message: Cannot access a disposed object.
Object name: 'System.Net.Sockets.NetworkStream'.

StackTrace:    at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.FtpClient.FtpSocketStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.Net.FtpClient.FtpSocketStream.WriteLine(Encoding encoding, String buf)
   at System.Net.FtpClient.FtpClient.Execute(String command)
   at System.Net.FtpClient.FtpClient.Disconnect()
   at System.Net.FtpClient.FtpClient.Dispose()
   at System.Net.FtpClient.FtpClient.Finalize()

Error 2:

Event Id : 1325

  Faulting application name: w3wp.exe, version: 8.5.9600.16384, time stamp: 0x52157ba0
    Faulting module name: KERNELBASE.dll, version: 6.3.9600.18666, time stamp: 0x58f32841
    Exception code: 0xe0434352
    Fault offset: 0x00015608
    Faulting process id: 0x37dc
    Faulting application start time: 0x01d35b56102b1ddf
    Faulting application path: C:\windows\SysWOW64\inetsrv\w3wp.exe
    Faulting module path: C:\windows\SYSTEM32\KERNELBASE.dll
    Report Id: b9f70358-c76a-11e7-8111-0cc47a0c4e39
    Faulting package full name: 
    Faulting package-relative application ID: 

At this moment, we were able to find which error is making app pool crashing. It concerns accessing disposed object from FtpClient library. But we were still not able to find what in our code is doing such this call.

Then, we decided to install DebugDiag ( https://www.microsoft.com/en-us/download/details.aspx?id=49924 ). A very powerful tool by the way. We launched DebugDiag for monitoring app pool crashes and we were able to generate a very detailed report (in .mhtml format).

Here is the full call stack : .NET Call Stack

[[HelperMethodFrame]] 
System_ni!System.Net.Sockets.NetworkStream.Write(Byte[], Int32, Int32)+673bde 
System.Net.FtpClient.FtpSocketStream.Write(Byte[], Int32, Int32)+33 
System.Net.FtpClient.FtpSocketStream.WriteLine(System.Text.Encoding, System.String)+44 
System.Net.FtpClient.FtpClient.Execute(System.String)+1c6 
System.Net.FtpClient.FtpClient.Disconnect()+57 
System.Net.FtpClient.FtpClient.Dispose()+46 
System.Net.FtpClient.FtpClient.Finalize()+10 
[[DebuggerU2MCatchHandlerFrame]] 
[[ContextTransitionFrame]] 
[[GCFrame]] 
[[DebuggerU2MCatchHandlerFrame]] 

Full Call Stack

KERNELBASE!RaiseException+48 
clr!RaiseTheExceptionInternalOnly+27c 
clr!IL_Throw+141 
[[HelperMethodFrame]] 
System_ni!System.Net.Sockets.NetworkStream.Write(Byte[], Int32, Int32)+673bde 
System.Net.FtpClient.FtpSocketStream.Write(Byte[], Int32, Int32)+33 
System.Net.FtpClient.FtpSocketStream.Write(Byte[], Int32, Int32)+33 
System.Net.FtpClient.FtpSocketStream.WriteLine(System.Text.Encoding, System.String)+44 
System.Net.FtpClient.FtpSocketStream.WriteLine(System.Text.Encoding, System.String)+44 
System.Net.FtpClient.FtpClient.Execute(System.String)+1c6 
System.Net.FtpClient.FtpClient.Disconnect()+57 
System.Net.FtpClient.FtpClient.Dispose()+46 
System.Net.FtpClient.FtpClient.Finalize()+10 
clr!MethodTable::FastBox+b0 
clr!MethodTable::CallFinalizer+139 
clr!CallFinalizer+a6 
clr!FinalizerThread::FinalizeAllObjects+a6 
clr!FinalizerThread::FinalizeAllObjects_Wrapper+14 
clr!Thread::DoExtraWorkForFinalizer+1b1 
clr!Thread::DoExtraWorkForFinalizer+234 
clr!Thread::DoExtraWorkForFinalizer+5f8 
[[DebuggerU2MCatchHandlerFrame]] 
clr!Thread::DoExtraWorkForFinalizer+137 
clr!Thread::DoADCallBack+30f 
[[ContextTransitionFrame]] 
clr!Thread::DoExtraWorkForFinalizer+19f 
clr!FinalizerThread::DoOneFinalization+129 
[[GCFrame]] 
clr!FinalizerThread::FinalizeAllObjects+a6 
clr!FinalizerThread::FinalizerThreadWorker+ed 
clr!Thread::DoExtraWorkForFinalizer+1b1 
clr!Thread::DoExtraWorkForFinalizer+234 
clr!Thread::DoExtraWorkForFinalizer+5f8 
[[DebuggerU2MCatchHandlerFrame]] 
clr!ManagedThreadBase::FinalizerBase+33 
clr!FinalizerThread::FinalizerThreadStart+d4 
clr!Thread::intermediateThreadProc+55 
kernel32!BaseThreadInitThunk+24 
ntdll!__RtlUserThreadStart+2f 
ntdll!_RtlUserThreadStart+1b 

We were able to find the exception that made the pool crashing but we are still unable to find which part of our code is doing the call. At the bottom of the call stack, you can noticed that the first event is : ntdll!_RtlUserThreadStart+1b which I really don't know ..

So I would be glad to have some help to solve this 2 questions :

• How to find in my code what is creating the bug ?
• How can such error making my application pool crashed and recycled ?

Environment : - IIS 8.5 - Application pool CLR version 4 - Windows Server 2012 R2

Answer 1

The message Cannot access a disposed object means that the garbage collector has disposed your object, which you then try to use.

Are you calling the FtpClient with using , are you doing a .close() ?

The error logs aren't actually useful, it would be better if you posted your code instead.

Also, put you code inside try...catch so that you can capture the error rather than leaving it unhandled. Sometimes file transfers fail so you need to be doing something like the following :

try { 
    using(Stream s = ftpClient.OpenRead()) { 
    // perform your transfer 
    } 
} 
catch(Exception) {
 // Typical exceptions here are IOException, SocketException, or a FtpCommandException 
}