How to minimize Logging impact

Question

There's already a question about this issue , but it's not telling me what I need to know: Let's assume I have a web application, and there's a lot of logging on every roundtrip. I don't want to open a debate about why there's so much logging, or how can I do less loggin operations. I want to know what possibilities I have in order to make this logging issue performant and clean .

So far, I've implemented declarative (attribute based) and imperative logging, which seems to be a cool and clean way of doing it... now, what can I do about performance , assuming I can expect those logs to take more time than expected. Is it ok to open a thread and leave that work to it?

Answer 1

Things I would consider:

• Use an efficient file format to minimise the quantity of data to be written (eg XML and text formats are easy to read but usually terribly inefficient - the same information can be stored in a binary format in a much smaller space). But don't spend lots of CPU time trying to pack data "optimally". Just go for a simple format that is compact but fast to write.

• Test use of compression on the logs. This may not be the case with a fast SSD but in most I/O situations the overhead of compressing data is less than the I/O overhead, so compression gives a net gain (although it is a compromise - raising CPU usage to lower I/O usage).

• Only log useful information. No matter how important you think everything is, it's likely you can find something to cut out.

• Eliminate repeated data. eg Are you logging a client's IP address or domain name repeatedly? Can these be reported once for a session and then not repeated? Or can you store them in a map file and use a compact index value whenever you need to reference them? etc

• Test whether buffering the logged data in RAM helps improve performance (eg writing a thousand 20 byte log records will mean 1,000 function calls and could cause a lot of disk seeking and other write overheads, while writing a single 20,000 byte block in one burst means only one function call and could give significant performance increase and maximise the burst rate you get out to disk). Often writing blocks in sizes like (4k, 16k, 32, 64k) of data works well as it tends to fit the disk and I/O architecture (but check your specific architecture for clues about what sizes might improve efficiency). The down side of a RAM buffer is that if there is a power outage you will lose more data. So you may have to balance performance against robustness.

• (Especially if you are buffering...) Dump the information to an in-memory data structure and pass it to another thread to stream it out to disk. This will help stop your primary thread being held up by log I/O. Take care with threads though - for example, you may have to consider how you will deal with times when you are creating data faster than it can be logged for short bursts - do you need to implement a queue, etc?

• Are you logging multiple streams? Can these be multiplexed into a single log to possibly reduce disk seeking and the number of open files?

• Is there a hardware solution that will give a large bang for your buck? eg Have you used SSD or RAID disks? Will dumping the data to a different server help or hinder? It may not always make much sense to spend $10,000 of developer time making something perform better if you can spend $500 to simply upgrade the disk.

Answer 2

I use the code below to Log. It is a singleton that accepts Logging and puts every message into a concurrentqueue. Every two seconds it writes all that has come in to the disk. Your app is now only delayed by the time it takes to put every message in the list. It's my own code, feel free to use it.

using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Windows.Forms;

namespace FastLibrary
{
    public enum Severity : byte
    {
        Info = 0,
        Error = 1,
        Debug = 2
    }

    public class Log
    {
        private struct LogMsg
        {
            public DateTime ReportedOn;
            public string Message;
            public Severity Seriousness;
        }

        // Nice and Threadsafe Singleton Instance
        private static Log _instance;

        public static Log File
        {
            get { return _instance; }
        }

        static Log()
        {
            _instance = new Log();
            _instance.Message("Started");
            _instance.Start("");
        }

         ~Log()
        {
            Exit();
        }

        public static void Exit()
        {
            if (_instance != null)
            {
                _instance.Message("Stopped");
                _instance.Stop();
                _instance = null;
            }
        }

        private ConcurrentQueue<LogMsg> _queue = new ConcurrentQueue<LogMsg>();
        private Thread _thread;
        private string _logFileName;
        private volatile bool _isRunning;

        public void Message(string msg)
        {
            _queue.Enqueue(new LogMsg { ReportedOn = DateTime.Now, Message = msg, Seriousness = Severity.Info });
        }

        public void Message(DateTime time, string msg)
        {
            _queue.Enqueue(new LogMsg { ReportedOn = time, Message = msg, Seriousness = Severity.Info });
        }

        public void Message(Severity seriousness, string msg)
        {
            _queue.Enqueue(new LogMsg { ReportedOn = DateTime.Now, Message = msg, Seriousness = seriousness });
        }

        public void Message(DateTime time, Severity seriousness, string msg)
        {
            _queue.Enqueue(new LogMsg { ReportedOn = time, Message = msg, Seriousness = seriousness });
        }

        private void Start(string fileName = "", bool oneLogPerProcess = false)
        {
            _isRunning = true;
            // Unique FileName with date in it. And ProcessId so the same process running twice will log to different files
            string lp = oneLogPerProcess ? "_" + System.Diagnostics.Process.GetCurrentProcess().Id : "";
            _logFileName = fileName == ""
                               ? DateTime.Now.Year.ToString("0000") + DateTime.Now.Month.ToString("00") +
                                 DateTime.Now.Day.ToString("00") + lp + "_" +
                                 System.IO.Path.GetFileNameWithoutExtension(Application.ExecutablePath) + ".log"
                               : fileName;
            _thread = new Thread(LogProcessor);
            _thread.IsBackground = true;
            _thread.Start();
        }

        public void Flush()
        {
            EmptyQueue();
        }

        private void EmptyQueue()
        {
            while (_queue.Any())
            {
                var strList = new List<string>();
                //
                try
                {
                    // Block concurrent writing to file due to flush commands from other context
                    lock (_queue)
                    {
                        LogMsg l;
                        while (_queue.TryDequeue(out l)) strList.Add(l.ReportedOn.ToLongTimeString() + "|" + l.Seriousness + "|" + l.Message);
                        if (strList.Count > 0)
                        {
                            System.IO.File.AppendAllLines(_logFileName, strList);
                            strList.Clear();
                        }
                    }
                }
                catch
                {
                    //ignore errors on errorlogging ;-)
                }
            }
        }

        public void LogProcessor()
        {
            while (_isRunning)
            {
                EmptyQueue();
                // Sleep while running so we write in efficient blocks
                if (_isRunning) Thread.Sleep(2000);
                else break;
            }
        }

        private void Stop()
        {
            // This is never called in the singleton.
            // But we made it a background thread so all will be killed anyway
            _isRunning = false;
            if (_thread != null)
            {
                _thread.Join(5000);
                _thread.Abort();
                _thread = null;
            }
        }
    }
}                                                

Answer 3

在调用logger.debug之前，请检查记录器是否已启用调试功能，这意味着在关闭调试时，您的代码不必评估消息字符串。

if (_logger.IsDebugEnabled) _logger.Debug($"slow old string {this.foo} {this.bar}");