Why does FileInfo Length not match the TotalFileSize passed to progress of CopyFileEx

Question

I am writing a backup utility. It is my first c# forms application.

For most files FileInfo length() matches the TotalFileSize passed to the CopyFileEx progress handler. However, for just one file Ihave found that FileInfo length() is less than the TotalFileSize passed to the progress handler.

The program uses FileInfo to calculate the total size of all files to be copied It uses CopyFileEx with a progress handler to copy files. The progress handler is used to update a progress bar for the proportion of all the data to be copied.

My problem is that sometimes the total number of bytes copied is greater than the total expected.

My investigation show that for most files FileInfo length() matches the TotalFileSize passed to the progress handler. However, for just one file FileInfo length() is less than the TotalFileSize passed to the progress handler.

Why do the sizes differ and what can I do to make the the total file size calculated match the total bytes copied?

    private UnsafeNativeMethods.CopyProgressResult localHandler(Int64 TotalFileSize, Int64 TotalBytesTransferred, Int64 StreamSize,
Int64 StreamBytesTransferred, UInt32 StreamNumber, UnsafeNativeMethods.CopyProgressCallbackReason CallbackReason, IntPtr SourceFile,
IntPtr DestinationFile, IntPtr Data)
    {
        switch (CallbackReason)
        {
            case UnsafeNativeMethods.CopyProgressCallbackReason.CallbackChunkedFinished:
                Debug.Print("localHandler: TotalBytesTransferred={0} TotalFileSize={1} ", TotalBytesTransferred.ToString(), TotalFileSize.ToString());
                break;
            case UnsafeNativeMethods.CopyProgressCallbackReason.CallbackStreamSwitch:
                break;
            default:
                break;
        }

        return UnsafeNativeMethods.CopyProgressResult.ProgressContinue;
    }

    private void ButtonTestFileSize_Click(object sender, EventArgs e)
    {
        bool success;
        bool b=false;
        string inputFile= @"C:\temp\WrongFileSize\myFile.conf";
        string outputFile= @"C:\temp\WrongFileSize\myFile.con2";


        /* Get the input Filename using FileInfo */
        FileInfo file = new FileInfo(inputFile);
        Debug.Print("input FileInfo.length={0}", file.Length);


        string hres = UnsafeNativeMethods.HResultToString(UnsafeNativeMethods.GetHResult((uint)Marshal.GetLastWin32Error()));
        success = UnsafeNativeMethods.CopyFileEx(inputFile,
                outputFile,
                new UnsafeNativeMethods.CopyProgressRoutine(localHandler),
                IntPtr.Zero,
                ref b,
                CopyFileFlags.FileFailIfExists | CopyFileFlags.COPY_FILE_NO_BUFFERING);
        if (!success)
        {
            Debug.Print("Failed");

        }
        else
        {

            Debug.Print("Success");
        }


        /* Get the output Filename using FileInfo */
        file = new FileInfo(outputFile);
        Debug.Print("outputFile FileInfo.length={0}", file.Length);

    }
}

The output from this code is as follows:

input FileInfo.length=2636
localHandler: TotalBytesTransferred=2636 TotalFileSize=2662 
localHandler: TotalBytesTransferred=2662 TotalFileSize=2662 
Success
outputFile FileInfo.length=2636

Answer 1

This is because you're copying more bytes than is in the primary stream of the file. There's a hint between the documentation for CopyFileEx's LPPROGRESS_ROUTINE :

The total number of bytes in the current stream that have been transferred from the source file to the destination file since the copy operation began.

and the output you witness:

input FileInfo.length=2636
localHandler: TotalBytesTransferred=2636 TotalFileSize=2662 
localHandler: TotalBytesTransferred=2662 TotalFileSize=2662 
Success
outputFile FileInfo.length=2636

If you added a message to your currently empty UnsafeNativeMethods.CopyProgressCallbackReason.CallbackStreamSwitch target, you'd more obviously see what's going on. Basically, you're transferring 2636 bytes for the primary stream. Most of the Win32 APIs that look at file size only report on the primary stream, and FileInfo() is no different here.

Then you switch to some secondary stream on the file, probably a :Zone.Identifier stream, since that's the common stream created when downloading from the internet, and copying the 26 bytes of that stream.

You can use the Streams tool from Sysinternals to view the different streams for a file, or use FindFirstStream / FindNextStream to enumerate them from code.