托管堆是否不可扩展到多核系统

Question

I was seeing some strange behavior in a multi threading application which I wrote and which was not scaling well across multiple cores.

The following code illustrates the behavior I am seeing. It appears the heap intensive operations do not scale across multiple cores rather they seem to slow down. ie using a single thread would be faster.

class Program
{
   public static Data _threadOneData = new Data();
   public static Data _threadTwoData = new Data();
   public static Data _threadThreeData = new Data();
   public static Data _threadFourData = new Data();

   static void Main(string[] args)
   {
      // Do heap intensive tests
      var start = DateTime.Now;
      RunOneThread(WorkerUsingHeap);
      var finish = DateTime.Now;
      var timeLapse = finish - start;
      Console.WriteLine("One thread using heap: " + timeLapse);

      start = DateTime.Now;
      RunFourThreads(WorkerUsingHeap);
      finish = DateTime.Now;
      timeLapse = finish - start;
      Console.WriteLine("Four threads using heap: " + timeLapse);

      // Do stack intensive tests
      start = DateTime.Now;
      RunOneThread(WorkerUsingStack);
      finish = DateTime.Now;
      timeLapse = finish - start;
      Console.WriteLine("One thread using stack: " + timeLapse);

      start = DateTime.Now;
      RunFourThreads(WorkerUsingStack);
      finish = DateTime.Now;
      timeLapse = finish - start;
      Console.WriteLine("Four threads using stack: " + timeLapse);

      Console.ReadLine();
   }

   public static void RunOneThread(ParameterizedThreadStart worker)
   {
      var threadOne = new Thread(worker);
      threadOne.Start(_threadOneData);

      threadOne.Join();
   }

   public static void RunFourThreads(ParameterizedThreadStart worker)
   {
      var threadOne = new Thread(worker);
      threadOne.Start(_threadOneData);

      var threadTwo = new Thread(worker);
      threadTwo.Start(_threadTwoData);

      var threadThree = new Thread(worker);
      threadThree.Start(_threadThreeData);

      var threadFour = new Thread(worker);
      threadFour.Start(_threadFourData);

      threadOne.Join();
      threadTwo.Join();
      threadThree.Join();
      threadFour.Join();
   }

   static void WorkerUsingHeap(object state)
   {
      var data = state as Data;
      for (int count = 0; count < 100000000; count++)
      {
         var property = data.Property;
         data.Property = property + 1;
      }
   }

   static void WorkerUsingStack(object state)
   {
      var data = state as Data;
      double dataOnStack = data.Property;
      for (int count = 0; count < 100000000; count++)
      {
         dataOnStack++;
      }
      data.Property = dataOnStack;
   }

   public class Data
   {
      public double Property
      {
         get;
         set;
      }
   }
}

This code was run on a Core 2 Quad (4 core system) with the following results:

One thread using heap: 00:00:01.8125000

Four threads using heap: 00:00:17.7500000

One thread using stack: 00:00:00.3437500

Four threads using stack: 00:00:00.3750000

So using the heap with four threads did 4 times the work but took almost 10 times as long. This means it would be twice as fast in this case to use only one thread??????

Using the stack was much more as expected.

I would like to know what is going on here. Can the heap only be written to from one thread at a time?

Answer 1

The answer is simple - run outside of Visual Studio...

I just copied your entire program, and ran it on my quad core system.

Inside VS (Release Build):

One thread using heap: 00:00:03.2206779
Four threads using heap: 00:00:23.1476850
One thread using stack: 00:00:00.3779622
Four threads using stack: 00:00:00.5219478

Outside VS (Release Build):

One thread using heap: 00:00:00.3899610
Four threads using heap: 00:00:00.4689531
One thread using stack: 00:00:00.1359864
Four threads using stack: 00:00:00.1409859

Note the difference. The extra time in the build outside VS is pretty much all due to the overhead of starting the threads. Your work in this case is too small to really test, and you're not using the high performance counters, so it's not a perfect test.

Main rule of thumb - always do perf. testing outside VS, ie: use Ctrl+F5 instead of F5 to run.

Answer 2

Aside from the debug-vs-release effects, there is something more you should be aware of.

You cannot effectively evaluate multi-threaded code for performance in 0.3s.

The point of threads is two-fold: effectively model parallel work in code, and effectively exploit parallel resources (cpus, cores).

You are trying to evaluate the latter. Given that thread start overhead is not vanishingly small in comparison to the interval over which you are timing, your measurement is immediately suspect. In most perf test trials, a significant warm up interval is appropriate. This may sound silly to you - it's a computer program fter all, not a lawnmower. But warm-up is absolutely imperative if you are really going to evaluate multi-thread performance. Caches get filled, pipelines fill up, pools get filled, GC generations get filled. The steady-state, continuous performance is what you would like to evaluate. For purposes of this exercise, the program behaves like a lawnmower.

You could say - Well, no, I don't want to evaluate the steady state performance. And if that is the case, then I would say that your scenario is very specialized. Most app scenarios, whether their designers explicitly realize it or not, need continuous, steady performance.

If you truly need the perf to be good only over a single 0.3s interval, you have found your answer. But be careful to not generalize the results.

If you want general results, you need to have reasonably long warm up intervals, and longer collection intervals. You might start at 20s/60s for those phases, but here is the key thing: you need to vary those intervals until you find the results converging. YMMV. The valid times vary depending on the application workload and the resources dedicated to it, obviously. You may find that a measurement interval of 120s is necessary for convergence, or you may find 40s is just fine. But (a) you won't know until you measure it, and (b) you can bet 0.3s is not long enough.

Answer 3

[edit]Turns out, this is a release vs. debug build issue -- not sure why it is, but it is. See comments and other answers.[/edit]

This was very interesting -- I wouldn't have guessed there'd be that much difference. (similar test machine here -- Core 2 Quad Q9300)

Here's an interesting comparison -- add a decent-sized additional element to the 'Data' class -- I changed it to this:

public class Data
{
    public double Property { get; set; }
    public byte[] Spacer = new byte[8096];
}

It's still not quite the same time, but it's very close (running it for 10x as long results in 13.1s vs. 17.6s on my machine).

If I had to guess, I'd speculate that it's related to cross-core cache coherency, at least if I'm remembering how CPU cache works. With the small version of 'Data', if a single cache line contains multiple instances of Data, the cores are having to constantly invalidate each other's caches (worst case if they're all on the same cache line). With the 'spacer' added, their memory addresses are sufficiently far enough apart that one CPU's write of a given address doesn't invalidate the caches of the other CPUs.

Another thing to note -- the 4 threads start nearly concurrently, but they don't finish at the same time -- another indication that there's cross-core issues at work here. Also, I'd guess that running on a multi-cpu machine of a different architecture would bring more interesting issues to light here.

I guess the lesson from this is that in a highly-concurrent scenario, if you're doing a bunch of work with a few small data structures, you should try to make sure they aren't all packed on top of each other in memory. Of course, there's really no way to make sure of that, but I'm guessing there are techniques (like adding spacers) that could be used to try to make it happen.

[edit] This was too interesting -- I couldn't put it down. To test this out further, I thought I'd try varying-sized spacers, and use an integer instead of a double to keep the object without any added spacers smaller.

class Program
{
    static void Main(string[] args)
    {
        Console.WriteLine("name\t1 thread\t4 threads");
        RunTest("no spacer", WorkerUsingHeap, () => new Data());

        var values = new int[] { -1, 0, 4, 8, 12, 16, 20 };
        foreach (var sv in values)
        {
            var v = sv;
            RunTest(string.Format(v == -1 ? "null spacer" : "{0}B spacer", v), WorkerUsingHeap, () => new DataWithSpacer(v));
        }

        Console.ReadLine();
    }

    public static void RunTest(string name, ParameterizedThreadStart worker, Func<object> fo)
    {
        var start = DateTime.UtcNow;
        RunOneThread(worker, fo);
        var middle = DateTime.UtcNow;
        RunFourThreads(worker, fo);
        var end = DateTime.UtcNow;

        Console.WriteLine("{0}\t{1}\t{2}", name, middle-start, end-middle);
    }

    public static void RunOneThread(ParameterizedThreadStart worker, Func<object> fo)
    {
        var data = fo();
        var threadOne = new Thread(worker);
        threadOne.Start(data);

        threadOne.Join();
    }

    public static void RunFourThreads(ParameterizedThreadStart worker, Func<object> fo)
    {
        var data1 = fo();
        var data2 = fo();
        var data3 = fo();
        var data4 = fo();

        var threadOne = new Thread(worker);
        threadOne.Start(data1);

        var threadTwo = new Thread(worker);
        threadTwo.Start(data2);

        var threadThree = new Thread(worker);
        threadThree.Start(data3);

        var threadFour = new Thread(worker);
        threadFour.Start(data4);

        threadOne.Join();
        threadTwo.Join();
        threadThree.Join();
        threadFour.Join();
    }

    static void WorkerUsingHeap(object state)
    {
        var data = state as Data;
        for (int count = 0; count < 500000000; count++)
        {
            var property = data.Property;
            data.Property = property + 1;
        }
    }

    public class Data
    {
        public int Property { get; set; }
    }
    public class DataWithSpacer : Data
    {
        public DataWithSpacer(int size) { Spacer = size == 0 ? null : new byte[size]; }
        public byte[] Spacer;
    }
}

Result:

1 thread vs. 4 threads

• no spacer 00:00:06.3480000 00:00:42.6260000
• null spacer 00:00:06.2300000 00:00:36.4030000
• 0B spacer 00:00:06.1920000 00:00:19.8460000
• 4B spacer 00:00:06.1870000 00:00:07.4150000
• 8B spacer 00:00:06.3750000 00:00:07.1260000
• 12B spacer 00:00:06.3420000 00:00:07.6930000
• 16B spacer 00:00:06.2250000 00:00:07.5530000
• 20B spacer 00:00:06.2170000 00:00:07.3670000

No spacer = 1/6th the speed, null spacer = 1/5th the speed, 0B spacer = 1/3th the speed, 4B spacer = full speed.

I don't know the full details of how the CLR allocates or aligns objects, so I can't speak to what these allocation patterns look like in real memory, but these definitely are some interesting results.