How do you pass an OutputStream as an InputStream in C#?

Question

My goal is to read from one stream, transform that stream, and use it as an input to a library that accepts a Stream to read.

I am using two different libraries. One takes an output Stream and transforms it. Let's call it TransformingOutputStream . Its intended use is:

var outputStream = new TransformingOutputStream(finalDestinationOutputStream);
inputStream.CopyTo(outputStream);

I'm using another library that accepts an input Stream . It does whatever it needs and then reads from that stream. Its intended use is:

MagicStreamReadingLibrary.ProcessStream(someInputStream);

I can't pass TransformingOutputStream to it because its intended use is to be written-to , not read-from. I do not have control over either library.

How do I hook-up the TransformingOutputStream to the library function that requires reading from an input Stream ?

Answer 1

So far this is the best working example I have, using Anonymous Pipes:

using( var pipeServer = new AnonymousPipeServerStream( PipeDirection.Out ) ) {
    var pipeServerTask = Task.Run(
        async () => {
            using( var stream = getInputStream() ) {
                await stream.CopyToAsync( new TransformingOutputStream( pipeServer ) );
            }
            pipeServer.WaitForPipeDrain();
            pipeServer.Dispose();
        } );

    using( var client = new AnonymousPipeClientStream( PipeDirection.In, pipeServer.ClientSafePipeHandle ) ) {
        MagicStreamReadingLibrary.ProcessStream( client );
    }
    pipeServerTask.Wait();
}

Answer 2

将其写入平面文件，然后将其读出。

Answer 3

Here is something I just threw together, it should in theory work (untested, I just know it compiles correctly).

public class BufferingStream
{
    private readonly Stream _readingStream;
    private readonly Stream _writingStream;
    private BlockingCollection<byte[]> _buffer;

    public BufferingStream()
    {
        _buffer = new BlockingCollection<byte[]>(new ConcurrentQueue<byte[]>());
        _readingStream = new InternalReadingStream(_buffer);
        _writingStream = new InternalWritingStream(_buffer);
    }

    public BufferingStream(int maxQueueLength)
    {
        _buffer = new BlockingCollection<byte[]>(new ConcurrentQueue<byte[]>(), maxQueueLength);
        _readingStream = new InternalReadingStream(_buffer);
        _writingStream = new InternalWritingStream(_buffer);
    }
    public Stream GetReadingStream()
    {
        return _readingStream;
    }

    public Stream GetWritingStream()
    {
        return _writingStream;
    }

    public int QueueLength
    {
        get { return _buffer.Count; }
    }

    public class InternalWritingStream : Stream
    {
        private readonly BlockingCollection<byte[]> _queuedBytes;

        public InternalWritingStream(BlockingCollection<byte[]> queuedBytes)
        {
            _queuedBytes = queuedBytes;
        }

        public override void Write(byte[] buffer, int offset, int count)
        {
            byte[] internalBuffer = new byte[count];
            Array.Copy(buffer, offset, internalBuffer, 0, count);
            _queuedBytes.Add(internalBuffer);
        }

        public override void Close()
        {
            _queuedBytes.CompleteAdding();
            base.Close();
        }

        public override void Flush()
        {
        }

        public override long Seek(long offset, SeekOrigin origin)
        {
            throw new NotSupportedException();
        }

        public override void SetLength(long value)
        {
            throw new NotSupportedException();
        }

        public override int Read(byte[] buffer, int offset, int count)
        {
            throw new NotSupportedException();
        }

        public override bool CanRead
        {
            get { return false; }
        }

        public override bool CanSeek
        {
            get { return false; }
        }

        public override bool CanWrite
        {
            get { return true; }
        }

        public override long Length
        {
            get { throw new NotSupportedException(); }
        }

        public override long Position
        {
            get { throw new NotSupportedException(); }
            set { throw new NotSupportedException(); }
        }
    }

    private sealed class InternalReadingStream : Stream
    {
        private readonly BlockingCollection<byte[]> _queuedBytes;
        private byte[] _currentItem;
        private int _currentItemOffset;

        public InternalReadingStream(BlockingCollection<byte[]> queuedBytes)
        {
            _queuedBytes = queuedBytes;
            _currentItem = new byte[0];
            _currentItemOffset = 0;
        }

        public override int Read(byte[] buffer, int offset, int count)
        {
            if (_currentItemOffset == _currentItem.Length)
            {
                //Try to take the next buffer, if we can't take a item it means we where done adding from the source.
                var taken = _queuedBytes.TryTake(out _currentItem, Timeout.Infinite);
                if (!taken)
                    return 0;

                _currentItemOffset = 0;
            }
            var bytesToRead = Math.Min(count, _currentItem.Length - _currentItemOffset);
            Array.Copy(_currentItem, _currentItemOffset, buffer, offset, bytesToRead);
            _currentItemOffset += bytesToRead;

            return bytesToRead;
        }

        public override void Flush()
        {
        }

        public override long Seek(long offset, SeekOrigin origin)
        {
            throw new NotSupportedException();
        }

        public override void SetLength(long value)
        {
            throw new NotSupportedException();
        }

        public override void Write(byte[] buffer, int offset, int count)
        {
            throw new NotSupportedException();
        }

        public override bool CanRead
        {
            get { return true; }
        }

        public override bool CanSeek
        {
            get { return false; }
        }

        public override bool CanWrite
        {
            get { return false; }
        }

        public override long Length
        {
            get { throw new NotSupportedException(); }
        }

        public override long Position
        {
            get { throw new NotSupportedException(); }
            set { throw new NotSupportedException(); }
        }
    }
}

The way it would work in your use case would be

var bufferingStream = new BufferingStream();

Task.Run(() =>
{
    using(var inputStream = GetTheStreamFromSomewhere();
    using(var finalDestinationOutputStream = bufferingStream.GetWritingStream())
    using(var outputStream = new TransformingOutputStream(finalDestinationOutputStream))
    {
        inputStream.CopyTo(outputStream);
    }
}

using(var someInputStream = bufferingStream.GetReadingStream()) //Technically a using is not necessary on the reading stream but it is good to keep good habits.
{
    MagicStreamReadingLibrary.ProcessStream(someInputStream);
}

Initially calls to .Read( calls ProcessStream makes will block until data becomes available. As bytes become available .Read( unblocks and passes along the data. Once finalDestinationOutputStream is disposed it will mark the queue as completed adding and once outputStream finishes its last read it will just return 0 for any subsequent calls.

If you find that your writer is much faster than your reader you may want to pass in a max queue length so writes will block till the reader has a chance to read.