C# 中的 Zlib 压缩和 MD5 散列

Question

I have a large file (Source file (assuming 10GB)), that I need to read it by chunks, compress and hash it. (Finally, we have two outputs: the hash of the file in string format (md5HashHex) and the compressed file in byte format (destData).) Also before compression, I need to add a header to the destination (destData) and hash it. After that, need to open the source file and read it chunk by chunk, compress and hash each chunk. I found out that my hashing would be different when I read the file chunk by chunk comparing to do the hash in one go. Here is my code, I appreciate if you can help me with that. Also I would like to know if I am doing the compression correctly. Thank you.

    public static void CompresingHashing(string inputFile)
    {
        MD5 md5 = MD5.Create();
        int byteCount = 0;
        var length = 8192;
        var chunk = new byte[length];
        byte[] destData;
        byte[] compressedData;
        byte[] header;
        header = Encoding.ASCII.GetBytes("HEADER");
        md5.TransformBlock(header, 0, header.Length, null, 0);
        destData = AppendingArrays(destData, header); //destination

        using (FileStream sourceFile = File.OpenRead(inputFile))
        {
          while ((byteCount = sourceFile.Read(chunk, 0, length)) > 0)
            {
              using (var ms = new MemoryStream())
              {
                using (ZlibStream result = new ZlibStream(ms, CompressionMode.Compress, CompressionLevel.Default)
                    result.Write(chunk, 0, chunk.Length);
              }
              compressedData = ms.ToArray();
              md5.TransformBlock(compressedData, 0, compressedData.Length, null, 0);
              destData = AppendingArrays(destData, compressedData);
            }
          md5.TransformFinalBlock(chunk, 0, 0);
          byte[] md5Hash = md5.Hash;
          string md5HashHex = string.Join(string.Empty, md5Hash.Select(b => b.ToString("x2")));
        }
        Console.WriteLine("Hash : " + hash);
    }

    public static byte[] AppendingArrays(byte[] existingArray, byte[] ArrayToAdd)
    {
        byte[] newArray = new byte[existingArray.Length + ArrayToAdd.Length];
        existingArray.CopyTo(newArray, 0);
        ArrayToAdd.CopyTo(newArray, existingArray.Length);
        return newArray;
    }

But If I hash destData (which is the source file + the header) I got the different result: (for the sake of space I didn't repeat the code )

.
.
.
destData = AppendingArrays(destData, compressedData);
byte[] md5Hash = md5.ComputeHash(data);
.
.
.

Answer 1

Looks like you are processing the last chunk twice on the md5. Simply call TransformFinalBlock with a byte[0] and length and offset of 0.