HttpWebResponse.GetResponseStream() (seems to be) failing with no exception
Question
I've been puzzling over this for a while. On one of our client (Windows 7, .NET 2.0) machines, the following code seems to be failing (causing the application to shut down) but it seems that no exceptions are thrown.
In the logs the 'Sending rest request' is printed then nothing. My expectation is that one of the following log messages would be printed (either one of the exceptions or one of the Info messages).
Going through the Event Log shows the Windows Error Report service starting then stopping about this time but there are no dump files created.
Any insight as to what could be happening or how to get more information to debug it would be greatly appreciated. Thanks
try
{
System.Net.ServicePointManager.Expect100Continue = false;
System.Net.ServicePointManager.UseNagleAlgorithm = false;
HttpWebRequest webRequest = ProxiedHttpWebRequest.CreateWithProxy(web_uri);
webRequest.Method = request_method;
if (content_type != null)
{
webRequest.ContentType = content_type;
}
webRequest.Method = request_method;
if (data != null)
{
webRequest.ContentLength = data.Length; //Count bytes to send
}
else
{
webRequest.ContentLength = 0;
}
webRequest.KeepAlive = true;
if (webRequest.Method == "POST")
{
//write post data
using (Stream os = webRequest.GetRequestStream())
{
os.Write(data, 0, data.Length);
}
}
if (headers != null)
{
foreach (KeyValuePair<string, string> pair in headers)
{
webRequest.Headers.Add(pair.Key, pair.Value);
}
}
Logger.Info("Sending rest request");
using (HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse())
{
if (webResponse != null)
{
Logger.Info("received response");
ResponseCode = webResponse.StatusCode;
using (StreamReader sr = new StreamReader(webResponse.GetResponseStream(), Encoding.UTF8))
{
string response = sr.ReadToEnd().Trim();
return response;
}
}
else
{
Logger.Info("a null response was received from the web");
}
}
return null;
}
catch (WebException we)
{
Logger.Warn(TraceLogger.GenerateErrorStringFromException(we));
throw we;
}
catch (Exception e)
{
Logger.Warn(TraceLogger.GenerateErrorStringFromException(e));
throw e;
}
EDIT: I should add this is a transient error (it fails once every couple of days and only on this machine). The code is running on 200+ machines
EDIT2: I also start the code with the following to catch unhanled exceptions:
// Add the event handler for handling non-UI thread exceptions to the event.
AppDomain.CurrentDomain.UnhandledException += CurrentDomain_UnhandledException;
// Add the event handler for handling UI thread exceptions to the event.
Application.ThreadException += Application_ThreadException;
// Set the unhandled exception mode to force all Windows Forms errors to go through
// our handler.
Application.SetUnhandledExceptionMode(UnhandledExceptionMode.CatchException);
EDIT3: Logging. It's definitely possible the logging code could be the cause of the crash. I'm using Log4Net from apache which is pretty stable and a RollingFileAppender which defaults to flush after each log request. The error also happens when the log level is such that the warnings wouldn't be getting logged.
EDIT4: After running procdump I was able to capture a dump of the program when it crashes. Opened it in Windbg and ran !analyze -v
FAULTING_IP:
+ae31be0
00000000`00000000 ?? ???
EXCEPTION_RECORD: ffffffffffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 0000000000000000
ExceptionCode: 80000003 (Break instruction exception)
ExceptionFlags: 00000000
NumberParameters: 0
FAULTING_THREAD: 0000000000001404
PROCESS_NAME: app_2_0_0_122.exe
ERROR_CODE: (NTSTATUS) 0x80000003 - {EXCEPTION} Breakpoint A breakpoint has been reached.
EXCEPTION_CODE: (HRESULT) 0x80000003 (2147483651) - One or more arguments are invalid
MOD_LIST: <ANALYSIS/>
NTGLOBALFLAG: 0
APPLICATION_VERIFIER_FLAGS: 0
MANAGED_STACK: !dumpstack -EE
No export dumpstack found
ADDITIONAL_DEBUG_TEXT: Followup set based on attribute [Is_ChosenCrashFollowupThread] from Frame:[0] on thread:[PSEUDO_THREAD]
LAST_CONTROL_TRANSFER: from 000007fef6ebce72 to 0000000076c0933a
DEFAULT_BUCKET_ID: STACKIMMUNE
PRIMARY_PROBLEM_CLASS: STACKIMMUNE
BUGCHECK_STR: APPLICATION_FAULT_STACKIMMUNE_NOSOS_WRONG_SYMBOLS
STACK_TEXT:
00000000`00000000 00000000`00000000 app_2_0_0+0x0
SYMBOL_NAME: app_2_0_0
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: app_2_0_0
IMAGE_NAME: app_2_0_0
DEBUG_FLR_IMAGE_TIMESTAMP: 0
STACK_COMMAND: ** Pseudo Context ** ; kb
FAILURE_BUCKET_ID: STACKIMMUNE_80000003_app_2_0_0!Unloaded
BUCKET_ID: X64_APPLICATION_FAULT_STACKIMMUNE_NOSOS_WRONG_SYMBOLS_app_2_0_0
EDIT: - output of WindDbg
0:000> !threads
- .. output removed and correct output below
0:000> !pe No export pe found
.. correct output below
0:000> !EEStack -EE No export EEStack found
- correct output below
After loading sos and clr and running ! analyze -v WindDbg also produced the following stack
STACK_TEXT:
00000000`0024e6d8 000007fe`f6ebce72 : 00000000`0024f000 00000000`0024e5d1 00000000`00000000 00000000`00000000 : user32!ZwUserWaitMessage+0xa
00000000`0024e6e0 000007fe`f0a1ec14 : 00000000`027f3260 00000000`0024e7c0 000007fe`f095f548 00000000`00000001 : mscorwks!DoNDirectCallWorker+0x62
00000000`0024e770 000007fe`f0a1e278 : 00000000`02857a00 00000000`00000001 00000000`ffffffff 00000000`00000000 : System_Windows_Forms_ni+0x35ec14
00000000`0024e9c0 000007fe`f0a1dce5 : 00000000`027f3260 00000000`ffffffff 00000000`02857998 000007fe`f104cd48 : System_Windows_Forms_ni+0x35e278
00000000`0024eb10 000007ff`001804a5 : 00000000`00000000 00000000`00000000 00000000`0024e888 00000000`0000000c : System_Windows_Forms_ni+0x35dce5
00000000`0024eb70 000007fe`f6ebd432 : 00000000`027f3668 00000000`00000000 000007ff`00033fa8 000007fe`f6d62fa9 : 0x7ff`001804a5
00000000`0024ebf0 000007fe`f6dab7a3 : 00000000`00000002 00000000`00000027 000007fe`f6c77340 00000000`00000000 : mscorwks!CallDescrWorker+0x82
00000000`0024ec40 000007fe`f729a981 : 00000000`0024ed78 00000000`00000000 00000000`0024ef80 00000000`00000008 : mscorwks!CallDescrWorkerWithHandler+0xd3
00000000`0024ece0 000007fe`f6e086db : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`0024f1e0 : mscorwks!MethodDesc::CallDescr+0x2b1
00000000`0024ef20 000007fe`f6e2bd54 : 00000000`00000000 00000000`00000000 0000000d`00d6000a 00000000`00000000 : mscorwks!ClassLoader::RunMain+0x22b
00000000`0024f180 000007fe`f7386c9d : 00000000`0024f7d0 00000000`00000000 00000000`0076bbc8 00000000`00000200 : mscorwks!Assembly::ExecuteMainMethod+0xbc
00000000`0024f470 000007fe`f6e3929f : 00000000`00000000 00000000`00000000 00000000`00000000 000007fe`f6e55c12 : mscorwks!SystemDomain::ExecuteMainMethod+0x47d
00000000`0024fa40 000007fe`f6e1bfdc : ffffffff`fffffffe 00000000`00000000 00009fee`00000000 00000000`00000000 : mscorwks!ExecuteEXE+0x47
00000000`0024fa90 000007fe`f7923309 : ffffffff`ffffffff 00000000`0075dbd0 00000000`00000000 00000000`0024fa98 : mscorwks!CorExeMain+0xac
00000000`0024faf0 000007fe`f7a15b21 : 000007fe`f6e1bf30 000007fe`f79232c0 00000000`00000000 00000000`00000000 : mscoreei!CorExeMain+0x41
00000000`0024fb20 00000000`7677652d : 000007fe`f7920000 00000000`00000000 00000000`00000000 00000000`00000000 : mscoree!CorExeMain_Exported+0x57
00000000`0024fb50 00000000`76e6c541 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!BaseThreadInitThunk+0xd
00000000`0024fb80 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x1d
EDIT: results of !pe below. Interestingly the dll it was trying to load exists. It is an auto generated serialization of the utilities dll used by both the service and application components.
0:000> !pe
Exception object: 0000000002831e48
Exception type: System.BadImageFormatException
Message: Could not load file or assembly 'Utilities_2_0_0_122.XmlSerializers, Version=2.0.0.122, Culture=neutral, PublicKeyToken=0bd8b79f92cc7463' or one of its dependencies. This assembly is built by a runtime newer than the currently loaded runtime and cannot be loaded.
InnerException: System.BadImageFormatException, use !PrintException 00000000028325c0 to see more
StackTrace (generated):
SP IP Function
000000000024E740 0000000000000001 System.Reflection.Assembly._nLoad(System.Reflection.AssemblyName, System.String, System.Security.Policy.Evidence, System.Reflection.Assembly, System.Threading.StackCrawlMark ByRef, Boolean, Boolean)
000000000024E740 000007FEF5DEBF61 System.Reflection.Assembly.InternalLoad(System.Reflection.AssemblyName, System.Security.Policy.Evidence, System.Threading.StackCrawlMark ByRef, Boolean)
000000000024E7D0 000007FEF5E249E4 System.Reflection.Assembly.Load(System.Reflection.AssemblyName)
000000000024E810 000007FEF42A5C0A System.Xml.Serialization.TempAssembly.LoadGeneratedAssembly(System.Type, System.String, System.Xml.Serialization.XmlSerializerImplementation ByRef)
StackTraceString: <none>
HResult: 8013101b
0:000> !PrintException 00000000028325c0
Exception object: 00000000028325c0
Exception type: System.BadImageFormatException
Message: Could not load file or assembly 'Utilities_2_0_0_122.XmlSerializers, Version=2.0.0.122, Culture=neutral, PublicKeyToken=0bd8b79f92cc7463' or one of its dependencies. This assembly is built by a runtime newer than the currently loaded runtime and cannot be loaded.
InnerException: <none>
StackTrace (generated):
<none>
StackTraceString: <none>
HResult: 8013101b
EDIT: As requested the source for the GenerateErrorStringFromException
public static string GenerateErrorStringFromException(Exception e)
{
try
{
string s = "";
s += "Outer Exception." + "\n";
s += "ExceptionType: " + e.GetType().Name + "\n";
s += "Message: " + e.Message + "\n";
s += "Source: " + e.Source + "\n";
s += "StackTrace: " + e.StackTrace + "\n";
s += "TargetSite: " + e.TargetSite + "\n";
string indent = "\t";
Exception ie = e;
while (ie.InnerException != null)
{
ie = ie.InnerException;
s += "Inner Exception." + "\n";
s += indent + "ExceptionType: " +
ie.GetType().Name + "\n";
s += indent + "Message: " + ie.Message + "\n";
s += indent + "Source: " + ie.Source + "\n";
s += indent + "StackTrace: " + ie.StackTrace + "\n";
s += indent + "TargetSite: " + ie.TargetSite + "\n";
indent += "\t";
}
return s;
}
catch (Exception)
{
return
"Fatal problem, an error was generated while printing an exception - no message given on orginal issue ";
}
}
EDIT: Should have included this earlier. Client machine is x64, Target build is Any_CPU. Client has .NET 4.5 installed, target platform is .NET 2.0.
EDIT: Ouput of !threads and !EEStack -EE
0:000> !threads
ThreadCount: 8
UnstartedThread: 0
BackgroundThread: 6
PendingThread: 0
DeadThread: 1
Hosted Runtime: no
PreEmptive Lock
ID OSID ThreadOBJ State GC GC Alloc Context Domain Count APT Exception
0 1 1404 000000000075dbd0 6020 Enabled 0000000000000000:0000000000000000 0000000000754c60 0 Ukn System.BadImageFormatException (0000000002831e48)
2 2 1540 0000000000763bd0 b220 Enabled 0000000000000000:0000000000000000 0000000000754c60 0 Ukn (Finalizer)
5 3 1ac0 000000001aedff20 880b220 Enabled 0000000002b11a50:0000000002b135c0 0000000000754c60 0 Ukn (Threadpool Completion Port)
6 4 1530 000000001aee0da0 80a220 Enabled 0000000000000000:0000000000000000 0000000000754c60 0 Ukn (Threadpool Completion Port)
8 6 41c 000000001c5d4960 180b220 Enabled 0000000000000000:0000000000000000 0000000000754c60 0 Ukn (Threadpool Worker) System.ArgumentException (0000000002a676d8)
10 5 dfc 000000001ba94600 180b220 Enabled 0000000002b19680:0000000002b1b5c0 0000000000754c60 2 Ukn (Threadpool Worker) System.ArgumentException (0000000002add658)
14 8 1e6c 000000001c9c2bf0 200b220 Enabled 0000000000000000:0000000000000000 0000000000754c60 1 Ukn
XXXX 7 0 000000001c9b69a0 9820 Enabled 0000000000000000:0000000000000000 0000000000754c60 0 Ukn
0:000> !EEEStack -EE
No export EEEStack found
0:000> !EEStack -EE
---------------------------------------------
Thread 0
Unable to load image C:\Windows\assembly\NativeImages_v2.0.50727_64\System.Windows.Forms\95674cb72317e3a5380ea450b913786f\System.Windows.Forms.ni.dll, Win32 error 0n2
*** WARNING: Unable to verify checksum for System.Windows.Forms.ni.dll
Child-SP RetAddr Call Site
000000000024e770 000007fef0a1e278 System_Windows_Forms_ni!System.Windows.Forms.Application+ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(Int32, Int32, Int32)+0x7d4
000000000024e9c0 000007fef0a1dce5 System_Windows_Forms_ni!System.Windows.Forms.Application+ThreadContext.RunMessageLoopInner(Int32, System.Windows.Forms.ApplicationContext)+0x578
000000000024eb10 000007ff001804a5 System_Windows_Forms_ni!System.Windows.Forms.Application+ThreadContext.RunMessageLoop(Int32, System.Windows.Forms.ApplicationContext)+0x65
000000000024eb70 000007fef6ebd432 app_2_0_0_122!app_2_0_0_122.exe!Unknown+0x245
---------------------------------------------
Thread 2
Child-SP RetAddr Call Site
---------------------------------------------
Thread 5
Child-SP RetAddr Call Site
---------------------------------------------
Thread 6
Child-SP RetAddr Call Site
---------------------------------------------
Thread 8
Child-SP RetAddr Call Site
---------------------------------------------
Thread 10
*** WARNING: Unable to verify checksum for mscorlib.ni.dll
Child-SP RetAddr Call Site
000000001d82e220 000007fef0a1e278 System_Windows_Forms_ni!System.Windows.Forms.Application+ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(Int32, Int32, Int32)+0x7d4
000000001d82e470 000007fef0a1dce5 System_Windows_Forms_ni!System.Windows.Forms.Application+ThreadContext.RunMessageLoopInner(Int32, System.Windows.Forms.ApplicationContext)+0x578
000000001d82e5c0 000007fef109f253 System_Windows_Forms_ni!System.Windows.Forms.Application+ThreadContext.RunMessageLoop(Int32, System.Windows.Forms.ApplicationContext)+0x65
000000001d82e620 000007ff001ac37d System_Windows_Forms_ni!System.Windows.Forms.Form.ShowDialog(System.Windows.Forms.IWin32Window)+0x763
000000001d82e910 000007ff001abce4 app_2_0_0_122!app_2_0_0_122.exe!Unknown+0x4dd
000000001d82ea00 000007ff001ab803 app_2_0_0_122!app_2_0_0_122.exe!Unknown+0x34
000000001d82ea50 000007ff001aaf9c app_2_0_0_122!app_2_0_0_122.exe!Unknown+0x113
000000001d82ead0 000007ff001a8706 app_2_0_0_122!app_2_0_0_122.exe!Unknown+0x23c
000000001d82eb70 000007ff001a85f8 app_2_0_0_122!app_2_0_0_122.exe!Unknown+0x96
000000001d82ebf0 000007fef5dedd38 Utilities_2_0_0_122!Utilities_2_0_0_122.dll!Unknown+0x58
000000001d82ec40 000007fef6ebd432 mscorlib_ni!System.Threading.ExecutionContext.runTryCode(System.Object)+0x178
000000001d82f4e0 000007fef5e2edd6 mscorlib_ni!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)+0x62
000000001d82f530 000007fef6ebd432 mscorlib_ni!System.Threading._TimerCallback.PerformTimerCallback(System.Object)+0x86
---------------------------------------------
Thread 14
Unable to load image C:\Windows\assembly\NativeImages_v2.0.50727_64\System\af0a0b96a02f9925eb84392ee65a5cfa\System.ni.dll, Win32 error 0n2
*** WARNING: Unable to verify checksum for System.ni.dll
Child-SP RetAddr Call Site
000000002202f090 000007fef5104487 mscorlib_ni!System.Threading.WaitHandle.WaitAny(System.Threading.WaitHandle[], Int32, Boolean)+0x64
000000002202f0f0 000007fef5dd2bbb System_ni!System.Net.TimerThread.ThreadProc()+0x327
000000002202f1c0 000007fef5e6aadd mscorlib_ni!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)+0x9b
000000002202f210 000007fef6ebd432 mscorlib_ni!System.Threading.ThreadHelper.ThreadStart()+0x4d
EDIT: procdump arguments procdump.exe -h -e -t app_2_0_0_122.exe C:\\Temp\\Dump.dump
and it also could be worth including the first line of WinDbg output, I assume it's the trigger for the dump *** Hung window detected: 180a5c
1 answers
solution1
1 2013-08-23 22:09:19
This line is strange:
ERROR_CODE: (NTSTATUS) 0x80000003 - {EXCEPTION} Breakpoint A breakpoint has been reached.
There is some Debbugger.Break or some Debug.Assert in the code of your logger?
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.
Related Question
HttpWebResponse.GetResponseStream converts < instead of < etc
Unable to assign HttpWebResponse.GetResponseStream() to Stream
HttpWebResponse.GetResponseStream(): when is the response body transmitted?
HttpWebResponse.GetResponseStream().Length throws exception even result is 200 OK status
Upload to S3 from HTTPWebResponse.GetResponseStream() in c#
HttpWebResponse GetResponseStream hanging in Dispose
Looking for a way to override the GetResponseStream Method of HttpWebResponse
System.Net.HttpWebResponse.GetResponseStream() returns truncated body in WebException
Receiving incomplete data when using HttpWebResponse::GetResponseStream method
correctly disposing HttpWebResponse in exception
Related Tags