System.DateTime Kind Bits

Question

Due to difficulties I experienced trying to call the DotNetOAuth CryptoKey constructor I started to investigate the .Net System.DateTime structure. According to what I've read, this object is actually represented by a 64 bit signed integer, with the "Ticks" encoded in the lower 62 bits and the Kind encoded in the upper 2 bits (IOW, it's a concatenation of the 2 bit Kind and 62 bit ticks).

Now I wanted to actually "see" this so I constructed a small C# program that created three System.DateTime objects as so:

    DateTime dtUtc = new System.DateTime(2014, 4, 29, 9, 10, 30, System.DateTimeKind.Utc);
    DateTime dtLocal = new System.DateTime(2014, 4, 29, 9, 10, 30, System.DateTimeKind.Local);
    DateTime dtU = new System.DateTime(2014, 4, 29, 9, 10, 30, System.DateTimeKind.Unspecified);

I then dumped the ticks property for each and, as expected, they were all equal. Finally, I applied .ToBinary()

    long bitUtc = dtUtc.ToBinary();
    long bitLocal = dtLocal.ToBinary();
    long bitU = dtU.ToBinary();

These longs were all different, again as expected. HOWEVER, I then tried to "inspect" the upper two bits to see which state corresponded to what settings, and found that the upper two bits were set the same in all three. I used the following routine to return the bit status:

public static bool IsBitSet<T>(this T t, int pos) where T : struct, IConvertible
{
    var value = t.ToInt64(CultureInfo.CurrentCulture);
    return (value & (1 << pos)) != 0;
}

(I got this from another post on SO), and called it like this:

    Boolean firstUtc = Class1.IsBitSet<long>(bitUtc, 63);
    Boolean secondUtc = Class1.IsBitSet<long>(bitUtc, 62);
    Boolean firstLocal = Class1.IsBitSet<long>(bitLocal, 63);
    Boolean secondLocal = Class1.IsBitSet<long>(bitLocal, 62);
    Boolean firstU = Class1.IsBitSet<long>(bitU, 63);
    Boolean secondU = Class1.IsBitSet<long>(bitU, 62);

Again, the first and second bits were set the same in all three (first was true, second false). I don't understand this, as I THOUGHT these would all be different, corresponding to the different SystemKind values.

Finally, I did some more reading and found (or at least it was said in one source) that MS doesn't serialize the Kind information in .ToBinary(). OK, but then why are the outputs of the .ToBinary() method all different?

I would appreciate info from anyone who could point me in the direction of a resource that would help me understand where I've gone wrong.

Answer 1

These longs were all different, again as expected. HOWEVER, I then tried to "inspect" the upper two bits to see which state corresponded to what settings, and found that the upper two bits were set the same in all three.

I really don't think that's the case - not with the results of ToBinary . Here's a short but complete program demonstrating the difference, using your source data, showing the results as hex (as if unsigned):

using System;

class Test
{
    static void Main()
    {
        DateTime dtUtc = new System.DateTime(2014, 4, 29, 9, 10, 30, System.DateTimeKind.Utc);
        DateTime dtLocal = new System.DateTime(2014, 4, 29, 9, 10, 30, System.DateTimeKind.Local);
        DateTime dtU = new System.DateTime(2014, 4, 29, 9, 10, 30, System.DateTimeKind.Unspecified);
        Console.WriteLine(dtUtc.ToBinary().ToString("X16"));
        Console.WriteLine(dtLocal.ToBinary().ToString("X16"));
        Console.WriteLine(dtU.ToBinary().ToString("X16"));
    }
}

Output:

48D131A200924700
88D131999ECDDF00
08D131A200924700

The top two bits are retrospectively 01, 10 and 00. The other bits change for the local case too, as per Marcin's post - but the top two bits really do indicate the kind.

The IsBitSet method is broken because it's left-shifting an int literal rather than a long literal. That means the shift will be mod 32, rather than mod 64 as intended. Try this instead:

public static bool IsBitSet<T>(this T t, int pos) where T : struct, IConvertible
{
    var value = t.ToInt64(CultureInfo.CurrentCulture);
    return (value & (1L << pos)) != 0;
}

Finally, I did some more reading and found (or at least it was said in one source) that MS doesn't serialize the Kind information in .ToBinary().

It's easy to demonstrate that's not true:

using System;

class Test
{
    static void Main()
    {
        DateTime start = DateTime.UtcNow;
        Show(DateTime.SpecifyKind(start, DateTimeKind.Utc));
        Show(DateTime.SpecifyKind(start, DateTimeKind.Local));
        Show(DateTime.SpecifyKind(start, DateTimeKind.Unspecified));
    }

    static void Show(DateTime dt)
    {
        Console.WriteLine(dt.Kind);
        DateTime dt2 = DateTime.FromBinary(dt.ToBinary());
        Console.WriteLine(dt2.Kind);
        Console.WriteLine("===");
    }
}

Answer 2

ToBinary() works differently for different DateTimeKind . You can see it on .NET source code :

public Int64 ToBinary() {
    if (Kind == DateTimeKind.Local) {
        // Local times need to be adjusted as you move from one time zone to another, 
        // just as they are when serializing in text. As such the format for local times
        // changes to store the ticks of the UTC time, but with flags that look like a 
        // local date.

        // To match serialization in text we need to be able to handle cases where
        // the UTC value would be out of range. Unused parts of the ticks range are
        // used for this, so that values just past max value are stored just past the
        // end of the maximum range, and values just below minimum value are stored
        // at the end of the ticks area, just below 2^62.
        TimeSpan offset = TimeZoneInfo.GetLocalUtcOffset(this, TimeZoneInfoOptions.NoThrowOnInvalidTime);
        Int64 ticks = Ticks;
        Int64 storedTicks = ticks - offset.Ticks;
        if (storedTicks < 0) {
            storedTicks = TicksCeiling + storedTicks;
        }
        return storedTicks | (unchecked((Int64) LocalMask));
    }
    else {
        return (Int64)dateData;
    }
}  

That's why you get different bits - local time is adjusted before transformed into bits, and so it does no longer match utc time.