How to solve producer/consumer race condition with BlockingCollection<>

Question

I am implementing a logger which writes records to the database. In order to prevent the database writes from blocking the code which is calling the logger, I've moved the DB access to a separate thread, implemented using a producer/consumer model based on BlockingCollection<string> .

Here's the simplified implementation:

abstract class DbLogger : TraceListener
{
    private readonly BlockingCollection<string> _buffer;
    private readonly Task _writerTask;

    DbLogger() 
    {
        this._buffer = new BlockingCollection<string>(new ConcurrentQueue<string>(), 1000);
        this._writerTask = Task.Factory.StartNew(this.ProcessBuffer, TaskCreationOptions.LongRunning);
    }

    // Enqueue the msg.
    public void LogMessage(string msg) { this._buffer.Add(msg); }

    private void ProcessBuffer()
    {
        foreach (string msg in this._buffer.GetConsumingEnumerable())
        {
            this.WriteToDb(msg);
        }
    }

    protected abstract void WriteToDb(string msg);

    protected override void Dispose(bool disposing) 
    { 
        if (disposing) 
        {
            // Signal to the blocking collection that the enumerator is done.
            this._buffer.CompleteAdding();

            // Wait for any in-progress writes to finish.
            this._writerTask.Wait(timeout);

            this._buffer.Dispose(); 
        }
        base.Dispose(disposing); 
    }
}

Now, when my application shuts down, I need to make sure that the buffer is flushed before the database connection goes down. Otherwise, WriteToDb will throw an exception.

So, here's my naive Flush implementation:

public void Flush()
{
    // Sleep until the buffer is empty.
    while(this._buffer.Count > 0)
    {
        Thread.Sleep(50);
    }
}

The problem with this implementation is the following sequence of events:

1. There is one entry in the buffer.
2. In the logging thread, MoveNext() is called on the enumerator, so we're now in the body of ProcessBuffer 's foreach loop.
3. Flush() is called by the main thread. It sees that the collection is empty, so returns immediately.
4. The main thread closes the DB connection.
5. Back in the logging thread, the body of the foreach loop starts executing. WriteToDb is called, and fails because the DB connection has been closed.

So, my next try was adding some flags, like so:

private volatile bool _isWritingBuffer = false;
private void ProcessBuffer()
{
    foreach (string msg in this._buffer.GetConsumingEnumerable())
    {
        lock (something) this._isWritingBuffer = true;
        this.WriteToDb(msg);
        lock (something) this._isWritingBuffer = false;
    }
}

public void Flush()
{
    // Sleep until the buffer is empty.
    bool isWritingBuffer;
    lock(something) isWritingBuffer = this._isWritingBuffer;
    while(this._buffer.Count > 0 || isWritingBuffer)
    {
        Thread.Sleep(50);
    }
}

However, there's still a race condition, since the entire Flush() method could execute after the collection is empty but before _isWritingBuffer gets set to true .

How can I fix my Flush implementation to avoid this race condition?

Note: For various reasons, I must write the logger from scratch, so please don't answer with a suggestion that I use some existing logging framework.

Answer 1

First never ever lock on a public object , especially this .

Moreover never ever use bare booleans for synchronization : see my blog if you want to have a glimpse in what can go wrong: Synchronization, memory visibility and leaky abstractions :)

Concerning the issue itself I must be missing something but why do you need such a Flush method?

Indeed when you're done with your logging you will dispose of the logger by calling its Dispose method from the main thread.

And you have implemented it in such a way that it will wait for the "write to DB" task.

If I'm wrong and you really need to synchronize with another primitive then you should use an event:

In the DbLogger :

public ManualResetEvent finalizing { get; set; }

public void Flush()
{
    finalizing.WaitOne();
}

And somewhere, eg in ProcessBuffer you notify when you're done with writing to DB:

finalizing.Set();