GZipStream decompression performance is poor

Question

I have a .NET 2.0 WinForms app that connects to a backend WAS server. I am using GZipStream to decode data coming back from a HttpWebRequest call made to the server. The data returned is compressed CSV, which Apache is compressing. The entire server stack is Hibernate-->EJB-->Spring-->Apache.

For small responses, the performance is fine (<50ms). When I get a response >150KB, it takes more than 60 seconds to decompress. The majority of the time seems to be spent in the GZipStream constructor.

This is the code showing where I get the response stream from the HttpWebResponse call:

using (Stream stream = this.Response.GetResponseStream())
{
 if (this.CompressData && this.Response.ContentEncoding == "gzip")
 {
        // Decompress the response
  byte[] b = Decompress(stream);
  this.ResponseBody = encoding.GetString(b);
    }
 else
 {
  // Just read the stream as a string
  using (StreamReader sr = new StreamReader(stream))
  {
   this.ResponseBody = sr.ReadToEnd();
  }
 }
}

Based on the comment from Lucero, I modified the Decompress method to the following, but I do not see any performance benefit from loading the ResponseStream into a MemoryStream before instantiating the GZipStream.

private static byte[] Decompress(Stream stream)
{
 using (MemoryStream ms = new MemoryStream())
 {
  byte[] buffer = new byte[4096];
  int read = 0;

  while ((read = stream.Read(buffer, 0, buffer.Length)) > 0)
  {
   ms.Write(buffer, 0, read);
  }

  ms.Seek(0, SeekOrigin.Begin);

  using (GZipStream gzipStream = new GZipStream(ms, CompressionMode.Decompress, false))
  {
   read = 0;
   buffer = new byte[4096];

   using (MemoryStream output = new MemoryStream())
   {
    while ((read = gzipStream.Read(buffer, 0, buffer.Length)) > 0)
    {
     output.Write(buffer, 0, read);
    }

    return output.ToArray();
   }
  }
 }
}

Based on the code above, can anyone see any issues? This seems quite basic to me, but it's driving me nuts.

I profiled the application using ANTS Profiler, and during the 60s of decompression, the CPU is near zero and the memory usage does not change.

The actual slowdown appears to be during the read of

this.Response.GetResponseStream

The entire 60s is spent loading the response stream into the MemoryStream. Once it's there, the call to GZipStream is quick.

I found that using HttpWebRequest.AutomaticDecompression exhibits the same performance issue, so I'm closing this question.

Answer 1

首先尝试将数据加载到MemoryStream中，然后解压缩MemoryStream ...

Answer 2

DotNetZip has a GZipStream class that can be used as a drop-in replacement for the System.IO.Compression.GZipStream.

DotNetZip is free.

NB: If you are only doing GZipStream, then you need the Ionic.Zlib.dll, not the Ionic.Zip.dll.

Answer 3

I'll drop my three cents to the subject, just to notify C#-users that a 7Zip seems to expose its API in plain C#. I think you all know the 7Zip tool quite well, and at least for me, regardless of how well- or ill- designed its API is --- knowing that is a big help in terms of better performance of handling ZIP files/streams.

ref: http://www.splinter.com.au/compressing-using-the-7zip-lzma-algorithm-in/

Answer 4

Sorry to not answer your question directly, but have you looked at SharpZip yet? I found it much easier to use than Gzip. If you have trouble solving your current problem, perhaps it would perform the task better.

http://www.icsharpcode.net/OpenSource/SharpZipLib/