log4net implementation detail - custom appender

Question

I've implemented a custom log4net appender that writes to an http service... works well, but I am suffering some premature optimization in my head. Specifically, is there a better way to do it? I guess I can make sure that only critical classes have that particular apprender, but it feels like that there could be a lot of appenders and a liability even with conservative logging options.

Does anyone have experience that they would like to share? I've looked at http://geekswithblogs.net/michaelstephenson/archive/2014/01/02/155044.aspx which is essentially what I am doing... (see code) How well does something like this scale? I like the factory for the singleton... what about implementing a concurrent queue to buffer the writes?

Hopefully I won't get spanked too hard by the admin for asking an (potentially opinion) best practice question.

(adding code from article for clarification)

    public class ServiceBusAppender : AppenderSkeleton
{
    public string ConnectionStringKey { get; set; }
    public string MessagingEntity { get; set; }
    public string ApplicationName { get; set; }
    public string EventType { get; set; }
    public bool Synchronous { get; set; }
    public string CorrelationIdPropertyName { get; set; }

    protected override void Append(log4net.Core.LoggingEvent loggingEvent)
    {
        var myLogEvent = new AzureLoggingEvent(loggingEvent);
        myLogEvent.ApplicationName = ApplicationName;
        myLogEvent.EventType = EventType;
        myLogEvent.CorrelationId = loggingEvent.LookupProperty(CorrelationIdPropertyName) as string;

        if (Synchronous)
            AppendInternal(myLogEvent, 0);
        else
        {
            Task.Run(() => AppendInternal(myLogEvent, 0));
        }            
    }

    protected void AppendInternal(AzureLoggingEvent myLogEvent, int attemptNo)
    {
        try
        {                
            //Convert event to JSON
            var stream = new MemoryStream();
            var json = Newtonsoft.Json.JsonConvert.SerializeObject(myLogEvent);
            var writer = new StreamWriter(stream);
            writer.Write(json);
            writer.Flush();
            stream.Seek(0, SeekOrigin.Begin);

            //Setup service bus message
            var message = new BrokeredMessage(stream, true);
            message.ContentType = "application/json";
            message.Label = myLogEvent.MessageType;                
            message.Properties.Add(new KeyValuePair<string, object>("ApplicationName", myLogEvent.ApplicationName));
            message.Properties.Add(new KeyValuePair<string, object>("UserName", myLogEvent.UserName));
            message.Properties.Add(new KeyValuePair<string, object>("MachineName", myLogEvent.MachineName));
            message.Properties.Add(new KeyValuePair<string, object>("MessageType", myLogEvent.MessageType));
            message.Properties.Add(new KeyValuePair<string, object>("Level", myLogEvent.Level));
            message.Properties.Add(new KeyValuePair<string, object>("EventType", myLogEvent.EventType));

            //Setup Service Bus Connection
            var connection = ConfigurationManager.ConnectionStrings[ConnectionStringKey];
            if (connection == null || string.IsNullOrEmpty(connection.ConnectionString))
            {
                ErrorHandler.Error("Cant publish the error, the connection string does not exist");
                return;
            }                
            var factory = MessagingFactoryManager.Instance.GetMessagingFactory(connection.ConnectionString);
            var sender = factory.CreateMessageSender(MessagingEntity);                

            //Publish
            sender.Send(message);                
        }
        catch (Exception ex)
        {
            if (ex.Message.Contains("The operation cannot be performed because the entity has been closed or aborted"))
            {
                if (attemptNo < 3)
                    AppendInternal(myLogEvent, attemptNo++);
                else
                    ErrorHandler.Error("Error occured while publishing error", ex);                    
            }
            else                
                ErrorHandler.Error("Error occured while publishing error", ex);                
        }
    }

    protected override void Append(log4net.Core.LoggingEvent[] loggingEvents)
    {
        foreach(var loggingEvent in loggingEvents)
        {
            Append(loggingEvent);
        }
    }        

Thx,

Chris

Answer 1

The cure for premature optimisation is to test and measure, then test and measure again. Write an integration test that logs to a thousand loggers, and see how that goes.

If that does show a problem, then rather than implement your own queue, inherit from BufferingAppenderSkeleton instead:

This base class should be used by appenders that need to buffer a number of events before logging them. For example the AdoNetAppender buffers events and then submits the entire contents of the buffer to the underlying database in one go.

Subclasses should override the SendBuffer method to deliver the buffered events.

The BufferingAppenderSkeleton maintains a fixed size cyclic buffer of events. The size of the buffer is set using the BufferSize property.

(As an aside, what is up with the log4net documentation, there seem to be more '½ï¿' characters every time I look at it?)

Answer 2

I see that your code involves JSON serialization. If you're looking for log4net JSON, why redo what has been done already? See log4net.ext.json . I'm the developer . The wiki covers the first steps on how to get it up and running. It is used in place of a layout so it can be plugged into any log4net appender that takes a layout.

Part of my project I have also created a load testing GUI for log4net. It is not released, but it should compile easily from source. You can use that to discover how different configurations scale in your conditions.

Finally, I'd advise to give LOCALHOST UDP delivery a shot if performance is priority. Projects like nxlog or logstash can swallow that easily. Again, why write new code?

Let me know if you need some clarification. Kind regards and good luck, Rob