如何获得启用了EWF（嵌入式Win 7）的硬盘驱动器或卷的一致SHA1

Question

Problem: I am unable to get reproducible file, volume or even physical drive hashes (eg SHA1) even with EWF enabled on Win 7 Embedded and using Win32 API's CreateFile/ReadFile methods. It appears (to me) that ReadFile is reading from the EWF's RAM overlay but I need it to get me the bytes from the disk.

Background: We are updating a regulated hardware+software product from Windows XP Embedded (FAT32) to Windows 7 Embedded (NTFS). The application code is written in C#.Net and VC++.

A regulatory requirement is to be able to verify that all files, including OS files, present on the hard disk are exact copies of the reference files that were given to the regulator. The regulator needs to be able to perform this verification while the hardware device is functional (in production) without stopping the applications running on the device.

In the XP Embedded days, we were able to get consistent file hashes when EWF was enabled, and with the device operational, by parsing the FAT32 file system and reading disk clusters via ReadFile (Win 32 API, kernel32.dll). The hashes were not consistent if we read the file as, say, a FileStream.

Our OS and applications are deployed as an immutable master disk image. We clone the hard drives from a master using a byte for byte drive clone hardware. We have also done the cloning via OSFClone (dd) on Linux.

We are trying to reproduce this (ie verifiable file hashes) in Windows 7 Embedded (x64) which requires an NTFS partition for the OS.

Test Environment:

• OS is Windows Embedded Standard 7 EWF is enabled in RAM mode on our volumes (C:, D:)
• NTFSLastAccessUpdate has been set to "1" in the registry under SYSTEM\\Control\\
• All Bootstat.dat files were deleted from the drive prior to enabling EWF and the changes committed via ewfmgr

Previously, to test whether the drive is immutable, I've done the following:

• Boot a device with the Win 7e drive and shutdown after making changes (EWF enabled)
• Plug the Win7e hard drive in a Kali Linux system and use dd | sha1sum to get the hash of the entire drive
• Plug back the Win7e drive in the device and boot, make changes (EWF enabled) and repeat the dd | sha1sum step on Kali Linux again (eg dd if=/dev/sda1 | sha1sum) where /dev/sda1 is an EWF protected Windows partition.

The signatures matched in that test between different boots. I'm running the test above again but it takes a while. [Edit: I have run the test again now: the EWF is working and Linux returns exact same hashes for /dev/sda1 between reboots of the Win7e drive]. NTFSLib and the test code pasted below do NOT reproduce same signatures of the EWF protected drive.

Problem: We've tried using CreateFile / ReadFile methods as well as an "NtfsLib" ( https://github.com/LordMike/NtfsLib ) to read individual files, volume and \\.\\PhysicalDriveN but we are not getting reproducible hashes between reboots of the device. That is, we get a different SHA1 hashes for a Windows file (C:\\windows\\windowsupdate.log) every time; we get different hashes for \\.\\C: everytime, and we get different hashes for \\.\\PhysicalDrive0 every time.

I'm pasting the C# code below that I'm using to calculate signatures. I don't mind re-writing the thing in another language, say C or C++, as long as I can get raw bytes from the hard disk. Please tell me if I'm doing something wrong in reading raw bytes. I do not need to read and hash individual files per se. I can read the entire disk or an entire volume and hash it as long as that hash matches between reboots of the device. Hashing the entire drive or the volume would satisfy the regulatory requirement. To me, it seems that the bytes I get from ReadFile show EWF's view of the filesystem.

I would appreciate any hints anyone can give me. I've been reading various posts about ReadFile but did not find any clues to this behavior with EWF.

I am running the code below as an Admin on Windows and my app.manifest has the requireAdmin thing set.

using System;
using System.Runtime.InteropServices;
using System.Security.Cryptography;
using Microsoft.Win32.SafeHandles;


namespace CSharpReadDisk
{
    class DiskReader
    {
        public const uint GenericRead = 0x80000000;
        public const uint FileShareRead = 1;
        public const uint FileShareWrite = 2;
        public const uint OpenExisting = 3;

        [DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
        static extern unsafe IntPtr CreateFile(string lpFileName, uint dwDesiredAccess,
          uint dwShareMode, IntPtr lpSecurityAttributes, uint dwCreationDisposition,
          uint dwFlagsAndAttributes, IntPtr hTemplateFile);

        [DllImport("kernel32.dll", SetLastError = true)]
        static extern unsafe bool ReadFile(IntPtr hFile, void* lpBuffer,
           uint nNumberOfBytesToRead, uint* lpNumberOfBytesRead, IntPtr lpOverlapped);

        [DllImport("kernel32", SetLastError = true)]
        static extern unsafe bool CloseHandle(IntPtr hObject);


        public unsafe IntPtr Open(string filename)
        {
            // open the existing file for reading       
            IntPtr handle = CreateFile(filename, GenericRead, FileShareRead | FileShareWrite, IntPtr.Zero, OpenExisting, 0, IntPtr.Zero);
            return handle;
        }

        public unsafe uint Read(IntPtr handle, byte[] buffer, uint count)
        {
            uint n = 0;
            fixed (byte* p = buffer)
            {
                if (!(ReadFile(handle, p, count, &n, IntPtr.Zero)))
                {
                    return 0;
                }
            }
            return n;
        }

        public unsafe bool Close(IntPtr handle)
        {
            return CloseHandle(handle);
        }
    }

    class Test
    { 
        static void Main(string[] args)
        {
            DiskReader dr = new DiskReader();

            Console.Write("Enter path to drive, volume or file: ");
            string path = Console.ReadLine();
            IntPtr fh = dr.Open(path);

            try
            { 
                SafeFileHandle sfh = new SafeFileHandle(fh, true);
                if (sfh.IsInvalid)
                {
                    Marshal.ThrowExceptionForHR(Marshal.GetHRForLastWin32Error());
                }

                Console.WriteLine("Enter read buffer size (MB): ");
                int bs = Console.Read();

                byte[] lpBuffer = new byte[bs * 1024 * 1024];

                uint bytesRead = 0;
                SHA1Managed sha1 = new SHA1Managed();
                while ((bytesRead = dr.Read(fh, lpBuffer, (uint)lpBuffer.Length)) > 0)
                {
                    sha1.TransformBlock(lpBuffer, 0, (int)bytesRead, null, 0);
                    Console.Write(".");
                }
                sha1.TransformFinalBlock(lpBuffer, 0, (int)bytesRead);
                Console.WriteLine("\nSHA1: {0}", BitConverter.ToString(sha1.Hash));
            }
            catch (Exception e)
            {
                Console.WriteLine("An exception occurred:\n HResult: {0}\n Message: {1}\n InnerException: {2}\n Source: {3}\n TargetSite: {4}\n StackTrace: {5}", 
                    e.HResult, e.Message, e.InnerException, e.Source, e.TargetSite, e.StackTrace);
            }
            dr.Close(fh); // close filehandle


            Console.WriteLine("\nPress any key to exit...");
            Console.ReadKey();
        }
    }
}

Answer 1

Update for anyone who is interested, you can actually go below EWF via CreateFile/ReadFile from Win32 API. You must use the \\.\\PhysicalDriveN handle and then read specific bytes that correspond to the volume you are interested in. You can do that by using the SetFilePointer function also provided by the Win32 API. Of course, if you wanted to hash the entire drive, you won't need to read specific bytes that correspond to the volume. MSDN docs on this subject are not very clear and the docs do not consider the specific EWF use case.