该进程由于未处理的异常而终止-WCF服务

Question

So I have a WCF service that i created that gets messages from customer and parses them to desired output and sends them to other customer via TCP/HTTP/FTP etc. This windows service has long running threads for each customer created using TPL's.

So for logging I have used NLOG, log to file and event viewer with below configurations

<target xsi:type="File" name="flatfile" layout="${longdate} ${uppercase:${level}} ${message} ${exception:format=tostring,StackTrace}" 
              archiveAboveSize="2000000" archiveFileName="${basedir}/logs/archive/${shortdate}-{#####}.engine.log" archiveNumbering="Sequence" archiveEvery="None" 
              maxArchiveFiles="100" fileName="${basedir}/logs/engine.current.log" keepFileOpen="true" concurrentWrites="true" />

<target xsi:type="EventLog" name="eventlog" layout="${longdate} ${uppercase:${level}} ${message} ${exception:format=tostring} ${StackTrace}" log="Application" source="Nuance Interface Engine Service" eventId="${event-properties:EventID}" />

Even when I added concurrentWrites="true", when I start WCF service after 20+ hours of time passes I got below error in event viewer

    Application: MyWcfService.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.OutOfMemoryException
   at System.Text.StringBuilder..ctor(System.String, Int32, Int32, Int32)
   at NLog.Layouts.SimpleLayout.GetFormattedMessage(NLog.LogEventInfo)
   at NLog.Targets.FileTarget.GetFormattedMessage(NLog.LogEventInfo)
   at NLog.Targets.FileTarget.GetBytesToWrite(NLog.LogEventInfo)
   at NLog.Targets.FileTarget.Write(NLog.Common.AsyncLogEventInfo[])
   at NLog.Targets.Target.WriteAsyncLogEvents(NLog.Common.AsyncLogEventInfo[])
   at NLog.Targets.Wrappers.AsyncTargetWrapper.ProcessPendingEvents(System.Object)
   at System.Threading.TimerQueueTimer.CallCallbackInContext(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.TimerQueueTimer.CallCallback()
   at System.Threading.TimerQueueTimer.Fire()
   at System.Threading.TimerQueue.FireQueuedTimerCompletion(System.Object)
   at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

Could you please someone guide me on where this issue is occuring, I thought multiple threads are accessing this log file hence I added concurrentWrites="true" attribute to nlog's file target.

after 1 second passes to above error, I see one more error in my event viewer.

Faulting application name: Hl7ic.Engine.View.exe, version: 18.0.1.160, time stamp: 0x5af5cd1f
Faulting module name: KERNELBASE.dll, version: 6.3.9600.18938, time stamp: 0x5a7dd8a7
Exception code: 0xe0434352
Fault offset: 0x00015ef8
Faulting process id: 0x1074
Faulting application start time: 0x01d3ea7338d9851c
Faulting application path: C:\Program Files (x86)\MyServices\MyWcfService.exe
Faulting module path: C:\windows\SYSTEM32\KERNELBASE.dll
Report Id: 59b36929-5688-11e8-80ca-005056a80aaa
Faulting package full name: 
Faulting package-relative application ID:

Answer 1

OutOfMemoryException means that there is not enough memory on the machine. Maybe it has been used up by another application. Maybe it has been used up by your application. Maybe your 32 bit application have used up its memory address space of 2 Gigabyte.

To diagnose this error, then use Process Explorer to capture a Fulldump of the application that uses too much memory. Then try using Visual Studio or WinDbg to investigate the memory dumpfile.

To remove NLog from the radar, then you can upgrade to NLog 4.4.6 (or newer). It will reduce memory allocations with 70 pct. from older versions (When using FileTarget)

If running NLog 4.4.6 (or newer) then you can optimize NLog even further by not using <targets async="true"> but instead use this default-wrapper :

<targets>
   <default-wrapper xsi:type="AsyncWrapper" timeToSleepBetweenBatches="0" />

   <target .... />

   <target .... />

</targets>

See also https://github.com/NLog/NLog/wiki/Performance