Chunked compressed response with NancyFX self hosting

Question

I have a system that is supposed to write lines to a HTTP response stream. Each line in this system represents some kind of event, so you can see this as a notification stream. I am using .NET4 on Windows 7 using NancyFX and Nancy self hosting (0.23). The following code is functional:

using System;
using System.IO;
using System.Threading;
using Nancy;
using Nancy.Hosting.Self;

namespace TestNancy
{
    public class ChunkedResponse : Response
    {
        public ChunkedResponse()
        {
            ContentType = "text/html; charset=utf-8";
            Contents = stream =>
            {
                using (var streamWriter = new StreamWriter(stream))
                {
                    while (true)
                    {
                        streamWriter.WriteLine("Hello");
                        streamWriter.Flush();
                        Thread.Sleep(1000);
                    }
                }
            };
        }
    }

    public class HomeModule : NancyModule
    {
        public HomeModule()
        {
            Get["/"] = args => new ChunkedResponse();
        }
    }

    public class Program
    {
        public static void Main()
        {
            using (var host = new NancyHost(new Uri("http://localhost:1234")))
            {
                host.Start();
                Console.ReadLine();
            }
        }
    }
}

Now I want to add compression to the stream to compress the amount of bandwidth. For some reason, when testing in a browser, I cannot see any result whatsoever. I have tried a lot of combinations to achieve the desired result, but this is what I have at the moment:

using System; using System.IO; using System.IO.Compression; using System.Threading; using Nancy; using Nancy.Hosting.Self;

namespace TestNancy {
    public class ChunkedResponse : Response
    {
        public ChunkedResponse()
        {
            Headers["Content-Encoding"] = "gzip";
            ContentType = "text/html; charset=utf-8";
            Contents = stream =>
            {
                using (var gzip = new GZipStream(stream, CompressionMode.Compress))
                using (var streamWriter = new StreamWriter(gzip))
                {
                    while (true)
                    {
                        streamWriter.WriteLine("Hello");
                        streamWriter.Flush();
                        Thread.Sleep(1000);
                    }
                }
            };
        }
    }

    public class HomeModule : NancyModule
    {
        public HomeModule()
        {
            Get["/"] = args => new ChunkedResponse();
        }
    }

    public class Program
    {
        public static void Main()
        {
            using (var host = new NancyHost(new Uri("http://localhost:1234")))
            {
                host.Start();
                Console.ReadLine();
            }
        }
    } }

I am looking for help that either tells me what I am doing wrong concerning the HTTP protocol (eg I tried adding chunk lengths as described in HTTP1.1, which did not work), or help concerning Nancy where it does something I did not account for.

Answer 1

The problem seems to be in Gzip implementation of the framework as it never writes to output stream before getting closed,

I simply used SharpZiplib and your code seems to work for me, here is my modifications

public class ChunkedResponse : Response
{
    public ChunkedResponse()
    {
        Headers["Transfer-Encoding"] = "chunked";
        Headers["Content-Encoding"] = "gzip";
        ContentType = "text/html; charset=utf-8";
        Contents = stream =>
        {
            var gzip = new ICSharpCode.SharpZipLib.GZip.GZipOutputStream(stream);
            using (var streamWriter = new StreamWriter(gzip))
            {
                while (true)
                {
                    streamWriter.WriteLine("Hello");
                    gzip.Flush();
                    streamWriter.Flush();
                    Thread.Sleep(1000);
                }
            }

        };
    }
}


public class HomeModule : NancyModule
{
    public HomeModule()
    {
        Get["/"] = args => new ChunkedResponse();
    }
}

public class Program
{
    public static void Main()
    {
        using (var host = new NancyHost(new HostConfiguration{AllowChunkedEncoding = true},new Uri("http://localhost:1234")))
        {

            host.Start();
            Console.ReadLine();
        }
    }
}

Nuget package for SharpZipLib: PM> Install-Package SharpZipLib

Answer 2

It looks as though whatever calls the delegate you supply as ChunkedReponse.Contents will never return because of the while(true) . Is this intended behaviour? Not knowing what this framework does with that delegate, I couldn't guess.

At first glance I did wonder whether the constructor would never return - which I guess would definitely cause a problem - but it didn't take me long to notice that it's a lambda. Fortunately.

Edit #1:

The documentation for GZipStream.Flush() says:

The current implementation of this method has no functionality. (Overrides Stream.Flush().)

This implies to me that GZipStream doesn't write anything to the transport until it's closed. Do you experience different behaviour if you don't run the mentioned delegate forever, and instead close the stream at some point?

Answer 3

I have tested it myself and I think the problem is how the browser handles these chunked and compressed responses. This is what I tried and what basically worked:

Contents = stream =>
        {
            using (var gzip = new GZipStream(stream, CompressionMode.Compress))
            using (var streamWriter = new StreamWriter(gzip))
            {
                for (int i = 0; i < 5;i++ )
                {
                    string txt = "Hello";
                    streamWriter.WriteLine(txt);
                    streamWriter.Flush();
                    Thread.Sleep(1000);
                }
            }
        };

The problem is that the browser waits for the chunked response to be ready before it displays the result. It probably waits with decompression until all data has been sent, although gzip supports streaming . Here is a first hint that supports my assumption.