Send a zip file as it is being created

Question

On my website, when a user clicks a certain button, a bunch of files have to be archived in a zip and sent out. The files themselves are generated by a third part and I only have the URLs. I have partly succeeded in that, but I have some issues.

First, if there are a lot of files to zip, the server response is slow as it first builds the zip file, then sends it. It can even crash after a while (notably, I get the error " Overflow or underflow in the arithmetic operation. ").

Second, right now the file only gets sent once the zip archive is complete. I would like the download to begin immediately instead. That is to say, as soon as the user has clicked "save" from the dialog, data begins sending and it keeps sending as the zip file is being created "on the fly". I have seen that feature on some websites, for example : http://download.muuto.com/

Problem is, I can't figure out how to do that.

I have used parts of code from this question : Creating a dynamic zip of a bunch of URLs on the fly And from this blog post : http://dejanstojanovic.net/aspnet/2015/march/generate-zip-file-on-the-fly-in-aspnet-mvc-application/

The zip file itself is created and returned in an ASP.NET MVC Controller method. Here is my code :

using ICSharpCode.SharpZipLib.Zip;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
using System.Web;
using System.Web.Mvc;

namespace MyProject.Controllers
{
    public class MyController : Controller
    {        
        public ActionResult DownloadFiles()
        {
            var files = SomeFunction();

            byte[] buffer = new byte[4096];

            var baseOutputStream = new MemoryStream();
            ZipOutputStream zipOutputStream = new ZipOutputStream(baseOutputStream);
            zipOutputStream.SetLevel(0); //0-9, 9 being the highest level of compression
            zipOutputStream.UseZip64 = UseZip64.Off;
            zipOutputStream.IsStreamOwner = false;

            foreach (var file in files)
            {
                using (WebClient wc = new WebClient())
                {
                    // We open the download stream of the file
                    using (Stream wcStream = wc.OpenRead(file.Url))
                    {
                        ZipEntry entry = new ZipEntry(ZipEntry.CleanName(file.FileName));
                        zipOutputStream.PutNextEntry(entry);

                        // As we read the stream, we add its content to the new zip entry
                        int count = wcStream.Read(buffer, 0, buffer.Length);
                        while (count > 0)
                        {
                            zipOutputStream.Write(buffer, 0, count);
                            count = wcStream.Read(buffer, 0, buffer.Length);
                            if (!Response.IsClientConnected)
                            {
                                break;
                            }
                        }
                    }
                }
            }
            zipOutputStream.Finish();
            zipOutputStream.Close();

            // Set position to 0 so that cient start reading of the stream from the begining
            baseOutputStream.Position = 0;

            // Set custom headers to force browser to download the file instad of trying to open it
            return new FileStreamResult(baseOutputStream, "application/x-zip-compressed")
            {
                FileDownloadName = "Archive.zip"
            };
        }
    }
}

Answer 1

Ok by fiddling a bit with the response outputstream and buffering, I've arrived to a solution :

using ICSharpCode.SharpZipLib.Zip;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
using System.Web;
using System.Web.Mvc;

namespace MyProject.Controllers
{
    public class MyController : Controller
    {        
        public ActionResult DownloadFiles()
        {
            var files = SomeFunction();

            // Disable Buffer Output to start the download immediately
            Response.BufferOutput = false;

            // Set custom headers to force browser to download the file instad of trying to open it
            Response.ContentType = "application/x-zip-compressed";
            Response.AppendHeader("content-disposition", "attachment; filename=Archive.zip");

            byte[] buffer = new byte[4096];

            ZipOutputStream zipOutputStream = new ZipOutputStream(Response.OutputStream);
            zipOutputStream.SetLevel(0); // No compression
            zipOutputStream.UseZip64 = UseZip64.Off;
            zipOutputStream.IsStreamOwner = false;

            try
            {
                foreach (var file in files)
                {
                    using (WebClient wc = new WebClient())
                    {
                        // We open the download stream of the image
                        using (Stream wcStream = wc.OpenRead(file.Url))
                        {
                            ZipEntry entry = new ZipEntry(ZipEntry.CleanName(file.FileName));
                            zipOutputStream.PutNextEntry(entry);

                            // As we read the stream, we add its content to the new zip entry
                            int count = wcStream.Read(buffer, 0, buffer.Length);
                            while (count > 0)
                            {
                                zipOutputStream.Write(buffer, 0, count);
                                count = wcStream.Read(buffer, 0, buffer.Length);
                                if (!Response.IsClientConnected)
                                {
                                    break;
                                }
                            }
                        }
                    }
                }
            }
            finally
            {
                zipOutputStream.Finish();
                zipOutputStream.Close();
            }

            return new HttpStatusCodeResult(HttpStatusCode.OK);
        }
    }
}