比较两个文件哪个更好？

Question

I have the following situation in C#:

ZipFile z1 = ZipFile.Read("f1.zip");
ZipFile z2 = ZipFile.Read("f2.zip");


MemoryStream ms1 = new MemoryStream();
MemoryStream ms2 = new MemoryStream()


ZipEntry zipentry1 = zip1["f1.dll"];
ZipEntry zipentry1 = zip2["f2.dll"];


zipentry1.Extract(ms1);
zipentry2.Extract(ms2);


byte[] b1 = new byte[ms1.Length];
byte[] b2 = new byte[ms2.Length];


ms1.Seek(0, SeekOrigin.Begin);
ms2.Seek(0, SeekOrigin.Begin);

what I have done here is opened 2 zip files f1.zip and f2.zip. Then I extract 2 files inside them (f1.txt and f2.txt inside f1.zip and f2.zip respectively) onto the MemoryStream objects. I now want to compare the files and find out if they are the same or not. I had 2 ways in mind:

1) Read the memory streams byte by byte and compare them. For this I would use

ms1.BeginRead(b1, 0, (int) ms1.Length, null, null);
ms2.BeginRead(b2, 0, (int) ms2.Length, null, null);

and then run a for loop and compare each byte in b1 and b2.

2) Get the string values for both the memory streams and then do a string compare. For this I would use

string str1 = Encoding.UTF8.GetString(ms1.GetBuffer(), 0, (int)ms1.Length);
string str2 = Encoding.UTF8.GetString(ms2.GetBuffer(), 0, (int)ms2.Length);

and then do a simple string compare.

Now, I know comparing byte by byte will always give me a correct result. But the problem with it is, it will take a lot time as I have to do this for thousands of files. That is why I am thinking about the string compare method which looks to find out if the files are equal or not very quickly. But I am not sure if string compare will give me the correct result as the files are either dlls or media files etc and will contain special characters for sure.

Can anyone tell me if the string compare method will work correctly or not ?

Thanks in advance.

PS : I am using DotNetLibrary.

Answer 1

The baseline for this question is the native way to compare arrays: Enumerable.SequenceEqual . You should use that unless you have good reason to do otherwise.

If you care about speed, you could attempt to p/invoke to memcmp in msvcrt.dll and compare the byte arrays that way. I find it hard to imagine that could be beaten. Obviously you'd do a comparison of the lengths first and only call memcmp if the two byte arrays had the same length.

The p/invoke looks like this:

[DllImport("msvcrt.dll", CallingConvention=CallingConvention.Cdecl)]
static extern int memcmp(byte[] lhs, byte[] rhs, UIntPtr count);

But you should only contemplate this if you really do care about speed, and the pure managed alternatives are too slow for you. So, do some timings to make sure you are not optimising prematurely. Well, even to make sure that you are optimising at all.

I'd be very surprised if converting to string was fast. I'd expect it to be slow. And in fact I'd expect your code to fail because there's no reason for your byte arrays to be valid UTF-8. Just forget you ever had that idea!

Answer 2

Compare ZipEntry.Crc and ZipEntry.UncompressedSize of the two files, only if they match uncompress and do the byte comparison. If the two files are the same, their CRC and Size will be the same too. This strategy will save you a ton of CPU cycles.

ZipEntry zipentry1 = zip1["f1.dll"];
ZipEntry zipentry2 = zip2["f2.dll"];

if (zipentry1.Crc == zipentry2.Crc && zipentry1.UncompressedSize == zipentry2.UncompressedSize)
{
    // uncompress
    zipentry1.Extract(ms1);
    zipentry2.Extract(ms2);

    byte[] b1 = new byte[ms1.Length];
    byte[] b2 = new byte[ms2.Length];

    ms1.Seek(0, SeekOrigin.Begin);
    ms2.Seek(0, SeekOrigin.Begin);

    ms1.BeginRead(b1, 0, (int) ms1.Length, null, null);
    ms2.BeginRead(b2, 0, (int) ms2.Length, null, null);

    // perform a byte comparison
    if (Enumerable.SequenceEqual(b1, b2)) // or a simple for loop
    {
        // files are the same
    }
    else
    {
        // files are different
    }
}
else
{
    // files are different
}