Log4net buffer doesn't work

Question

I am using log4net together with log4net.Elasticsearch plugin (this shoudn't matter, question is about log4net buffering in general) and I am trying to use buffer. I don't want every log event to be send to Elasticsearch separately. This is my configuration of appender:

<appender name="ElasticSearchAppender" type="log4net.ElasticSearch.ElasticSearchAppender, log4net.ElasticSearch">
    <layout type="log4net.Layout.PatternLayout,log4net">
      <param name="ConversionPattern" value="%d{ABSOLUTE} %-5p %c{1}:%L - %m%n" />
    </layout>
    <connectionString value="Server=xxx.xxx.xxx.xxx;Index=api_actions_log;Port=9200"/>
    <lossy value="false" />
    <bufferSize value="100" />
    <evaluator type="log4net.Core.LevelEvaluator">
        <threshold value="INFO"/>
    </evaluator>
    <filter type="log4net.Filter.LoggerMatchFilter">
          <loggerToMatch value="My.Namespace.LoggingFilterAttribute" />
    </filter>
    <filter type="log4net.Filter.DenyAllFilter" />
</appender>

As you can see I specify bufferSize=100 but even that every single message is send immediately to Elasticsearch.

I was digging into and trying to find out what's really happening in code. When Log method is called to goes down to Append method on ES appender class. Method is inherited from base log4net BufferingAppenderSkeleton and looks like this:

protected override void Append(LoggingEvent loggingEvent)
{
  if (this.m_cb == null || this.m_bufferSize <= 1)
  {
    if (this.m_lossy && (this.m_evaluator == null || !this.m_evaluator.IsTriggeringEvent(loggingEvent)) && (this.m_lossyEvaluator == null || !this.m_lossyEvaluator.IsTriggeringEvent(loggingEvent)))
      return;
    if (this.m_eventMustBeFixed)
      loggingEvent.Fix = this.Fix;
    this.SendBuffer(new LoggingEvent[1]
    {
      loggingEvent
    });
  }
  else
  {
    loggingEvent.Fix = this.Fix;
    LoggingEvent loggingEvent1 = this.m_cb.Append(loggingEvent);
    if (loggingEvent1 != null)
    {
      if (!this.m_lossy)
      {
        this.SendFromBuffer(loggingEvent1, this.m_cb);
      }
      else
      {
        if (this.m_lossyEvaluator == null || !this.m_lossyEvaluator.IsTriggeringEvent(loggingEvent1))
          loggingEvent1 = (LoggingEvent) null;
        if (this.m_evaluator != null && this.m_evaluator.IsTriggeringEvent(loggingEvent))
          this.SendFromBuffer(loggingEvent1, this.m_cb);
        else if (loggingEvent1 != null)
          this.SendBuffer(new LoggingEvent[1]
          {
            loggingEvent1
          });
      }
    }
    else if (this.m_evaluator != null && this.m_evaluator.IsTriggeringEvent(loggingEvent))
      this.SendFromBuffer((LoggingEvent) null, this.m_cb);
  }
}

At the beginning of the method it first check if we use buffer. Answer is yes so it goes into else block of outer If. Then it adds event to buffer and check if any event has been returned from buffer:

    LoggingEvent loggingEvent1 = this.m_cb.Append(loggingEvent);
    if (loggingEvent1 != null)

If is evaluated as false so code continues with the else if:

    else if (this.m_evaluator != null && this.m_evaluator.IsTriggeringEvent(loggingEvent))
      this.SendFromBuffer((LoggingEvent) null, this.m_cb);

Condition is evaluated as true because I have evaluator in my log4net config (level evaluator) and message is on proper level. SendFromBuffer method immediately sends message to storage (Elasticsearch in my case).

After this breakdown into log4net code I don't understand how buffering can work or what I need to setup. It looks to me that there will be always some evaluator (at least level evaluator) so SendFromBuffer method will be always called causing every single message to be send to storage separately. I would like to know how to setup log4net to really use buffer.

Answer 1

log4net is working as documented : the evaluator property "gets or sets the ITriggeringEventEvaluator that causes the buffer to be sent immediately ."

From this explanation :

The Evaluator is a pluggable object that is used by the BufferingAppenderSkeleton to determine if a logging event should not be buffered, but instead written/sent immediately. If the Evaluator decides that the event is important then the whole contents of the current buffer will be sent along with the event.

Typically an SmtpAppender will be setup to buffer events before sending as the cost of sending an email may be relatively high. If an important event arrives, say an ERROR, we would like this to be delivered immediately rather than waiting for the buffer to become full. This is where the Evaluator comes in as it allows us to say: "when an important event arrives don't worry about buffering, just send over everything you have right now".

So every message that has a log lovel of Info or higher will trigger your LevelEvaluator and cause the buffer to be sent.