Thread-safe async byte queue

Question

I've got a callback method that is called whenever new data is available:

public delegate void DataCallback(
    byte[] buffer,
    int offset,
    int count);

I want to wrap this in a class that implements an interface similar to this:

public interface IDataSource
{
    IAsyncResult BeginRead(
        byte[] buffer,
        int offset,
        int size,
        TimeSpan timeout,
        AsyncCallback callback,
        object state);

    int EndRead(
        IAsyncResult asyncResult);

    int Read(
        byte[] buffer,
        int offset,
        int size,
        TimeSpan timeout);
}

This is obviously a classical producer-consumer problem: the bytes are produced by calls to the callback method, and consumed by the Begin/EndRead and Read methods. The Begin/EndRead and Read methods should block if no data is available (until a timeout occurs). The implementation should use a fixed-size internal buffer, so the callback method needs to block when the buffer is currently full.

Since thinking about multithreading usually results in a severe headache, my question is: Is there already an implementation of such a data structure?

(I think implementing the Read method should be quite simple, but I'd like to avoid implementing Begin/EndRead with Read. Begin / EndInvoke .)

Answer 1

Does it have to be async via IAsyncResult ? I have a generic blocking queue here (ie readers block until there is data or it is closed; writers block until there is space); it isn't optimised specifically for byte[] , but as long as the size isn't vast it should cope - but as a blocking queue it requires (at least one) dedicated consumer thread, doing:

T val;
while(queue.TryDequeue(out val)) {
    // process val
}

Answer 2

I think you should do a google search on "lockless queue". I got lots of usefull hits that way.