How to handle Exceptions in a TPL Dataflow - Producer/Consumer

Question

I have the following test application that simulates my application scenario:

class Program
{
    static void Main()
    {
        Console.WriteLine("***Press R = record to new file, S = Stop Recording, E = Exit");
        var timer = new Timer(Callback, null, 0, 1000);
        while(!_Close) HandleInput(Console.ReadLine());
        Console.WriteLine("Finished");
    }

    private static bool _Close;

    private static void HandleInput(string input)
    {
        switch (input.ToLower())
        {
            case "r": CreateWriter();
                break;
            case "s": Console.WriteLine("File Closed: {0}", _FileWriter.Name); 
                _FileWriter.Dispose();
                _FileWriter = null;
                break;
            case "e":
                _Close = true;
                break;
        }
    }

    private static void CreateWriter()
    {
        if (_FileWriter != null)
            _FileWriter.Dispose();
        string filename = Path.Combine("C:\\", string.Format("{0:yyyy_MM_dd HH_mm_ss}.txt",DateTime.Now));
        _FileWriter = new AsyncFileWriter(filename);
        Console.WriteLine("New File Created: {0}", filename);
    }

    private static void Callback(object state)
    {
        if (_FileWriter != null)
            _FileWriter.Produce(MakeData());
    }

    private static byte[] MakeData()
    {
        string data = string.Empty;
        for (int i = 0; i < 50; i++)
        {
            data += string.Format("{0:yyyy-MM-dd HH:mm:ss.fff}{1}", DateTime.Now, Environment.NewLine);
        }
        return Encoding.UTF8.GetBytes(data);
    }

    private static AsyncFileWriter _FileWriter;
}

It uses the following class as the Consumer(/producer):

public class AsyncFileWriter : IDisposable
{
    private readonly FileStream _Filewriter;
    private readonly Task _WriteTask;
    private readonly BufferBlock<byte[]> _BufferBlock;

    public AsyncFileWriter(string filename)
    {
        _Filewriter = new FileStream(filename, FileMode.CreateNew, FileAccess.Write, FileShare.None);
        _BufferBlock = new BufferBlock<byte[]>();
        _WriteTask = WriteToFile();
    }

    public void Produce(byte[] data)
    {
        _BufferBlock.Post(data);
    }

    public long Filesize { get; private set; }

    public string Name { get { return _Filewriter.Name; } }

    private async Task WriteToFile()
    {
        while (await _BufferBlock.OutputAvailableAsync())
        {
            byte[] data = _BufferBlock.Receive();
            await _Filewriter.WriteAsync(data, 0, data.Length);
            Filesize = _Filewriter.Length;
        }
    }

    private async Task Complete()
    {
        _BufferBlock.Complete();
        await Task.WhenAll(_WriteTask, _BufferBlock.Completion);
        //now close the file
        _Filewriter.Dispose();
    }

    public void Dispose()
    {
        Complete();
    }
}

The important things to note are:

• I have a continuous stream of bytes that are passed to me from a proprietary library via a callback. I do not know when I will receive this data.
• I need to be able to control when to start recording this data (press "R")
• I need to be able to switch to a new file, but ensure previous data has been written to the last file (press "R" again while already recording)

I have a few questions, as I am quite new to the TPL dataflow library.

1. Firstly, is my implementation of Producer/Consumer (ie AsyncFileWriter) sound?
2. Is my implementation of IDisposable OK?
3. How do I handle exceptions in the Consumer (ie WriteToFile)? I really need to be notified if something goes wrong, but as you can see the only public method is Produce() , and the exception won't happen here... Is there a best practise for this pattern?

Answer 1

Dispose is kind of broken because it does not wait until all activity is shut down. This is probably not what you want. See http://blog.stephencleary.com/2013/03/async-oop-6-disposal.html .

AsyncFileWriter has an unbounded buffer. If the file writer is overloaded this will consume more and more memory.

AsyncFileWriter seems OK otherwise.

But since the file system does its own buffering you maybe can get away with not using anything like AsyncFileWriter at all. Just writing to the file might be good enough.

Regarding exceptions: You can expose an event for that. Or, in case of an exception you shut down writing, remember the exception and make it possible for consumers of the class to examine the exception. Also, Produce must start to fail. Dispose should not fail but before disposing users of the class need to shut it down and check for exceptions.

So either use an event or move the class into an error state.

Probably, the error state is a little nicer for consumers because the exception is discovered at their leisure, not pushed through an event. Complete should be public. The caller should await it and the exception should be thrown here.