MemoryStream VS FileStream which one is better to use in Webservice for GZipStream decompress?

Question

I have a asp.net webservice. some functionality of this webservice is to decompress clients request first. for this i have wrote 2 methods one uses MemoryStream and other uses FileStream .

When using MemoryStream sometiomes it get OutofMemoryException . so i have planned to use FileStream instead of MemoryStream for this reason.

before using this i just need a clarification that i am doing the right thing for the right job.

N:B: Sometiome my clients will send 10MB+ data to the webservice which i need to decompress in webservice side. i have more then 200 clients running. and where the webservice is hosted there are more then 30 web application and webservice also hosted though my webservice is under different app pool .

I did see : GZIP decompression C# OutOfMemory

i get some knowledge from there but for webservice which one should better i have quite confusion based on my situation. and i need a clear understanding about this.

Decompress method (uses MemoryStream) is below:

public static string Decompress(string compressedText)
        {
            try
            {
                byte[] gzBuffer = Convert.FromBase64String(compressedText);
                using (MemoryStream ms = new MemoryStream())
                {
                    int msgLength = BitConverter.ToInt32(gzBuffer, 0);
                    ms.Write(gzBuffer, 4, gzBuffer.Length - 4);

                    byte[] buffer = new byte[msgLength];

                    ms.Position = 0;
                    using (GZipStream zip = new GZipStream(ms, CompressionMode.Decompress))
                    {
                        zip.Read(buffer, 0, buffer.Length);
                    }

                    return Encoding.UTF8.GetString(buffer);
                }
            }
            catch (Exception ex)
            {
                DataSyncLog.Debug(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType + "::" + System.Reflection.MethodBase.GetCurrentMethod().ToString() + ":" + ex.ToString()+" : "+ex.StackTrace);
            }

            return string.Empty;
        }

Decompress method (uses FileStream) is below:

public static string Decompress(string compressedText)
        {
            string SourceDirectory = System.Guid.NewGuid().ToString();
            string DestinationDirectory = System.Guid.NewGuid().ToString();
            try
            {
                File.WriteAllBytes(SourceDirectory, Convert.FromBase64String(compressedText));
                using (FileStream fd = File.Create(DestinationDirectory))
                {
                    using (FileStream fs = File.OpenRead(SourceDirectory))
                    {
                        fs.Seek(4, 0);

                        using (Stream csStream = new GZipStream(fs, CompressionMode.Decompress))
                        {

                            byte[] buffer = new byte[1024];

                            int nRead;

                            while ((nRead = csStream.Read(buffer, 0, buffer.Length)) > 0)
                            {
                                fd.Write(buffer, 0, nRead);
                            }
                        }
                    }
                }

                return Encoding.UTF8.GetString(File.ReadAllBytes(DestinationDirectory));
            }
            catch (Exception ex)
            {
                DataSyncLog.Debug(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType + "::" + System.Reflection.MethodBase.GetCurrentMethod().ToString() + ":" + ex.ToString() + " : " + ex.StackTrace);
                return string.Empty;
            }
            finally
            {
                ClearFiles(SourceDirectory);
                ClearFiles(DestinationDirectory);
            }
        }

Someone could you please show me the right direction that which one should i use or any modification needed to the method that uses MemoryStream that can overcome this error. i will be grateful to you if you give me a clear understanding about this or any code change suggestion.

Answer 1

Working with stream looks more efficient memory-wise in the second case: with memory stream you hold entire stream in memory, with file stream you have only buffer of limited size.

Both of your methods can have problems with memory just because of their signature: when client sends 10MB this amount of memory will be allocated for compressedText argument and for return value.

You can look into changing interface of your service, so data is transfered in chunks (here you can find example of similar approach - http://www.codeproject.com/Articles/43272/Uploading-Large-Files-Through-Web-Service )

Or, if you can consider switching to WCF, it supports streaming transfer mode - http://msdn.microsoft.com/en-us/library/ms751463.aspx