并行执行比串行执行需要更多时间？

Question

i am studying task implementation in TBB and have run code for parallel and serial calculation of Fibonacci Series.

The Code is :

#include <iostream>
#include <list>
#include <tbb/task.h>
#include <tbb/task_group.h>
#include <stdlib.h>
#include "tbb/compat/thread"
#include "tbb/task_scheduler_init.h"
using namespace std;
using namespace tbb;

#define CutOff 2

long serialFib( long n ) {
if( n<2 )
return n;
else
return serialFib(n-1) + serialFib(n-2);
}


class FibTask: public task 
{
    public:
    const long n;
    long* const sum;

    FibTask( long n_, long* sum_ ) : n(n_), sum(sum_) {}

    task* execute() 
    {
        // cout<<"task id of thread is \t"<<this_thread::get_id()<<"FibTask(n)="<<n<<endl;  // Overrides virtual function task::execute    
                // cout<<"Task Stolen is"<<is_stolen_task()<<endl;
        if( n<CutOff ) 
        {
            *sum = serialFib(n);
        }
         else
         {
            long x, y;
            FibTask& a = *new( allocate_child() ) FibTask(n-1,&x);
            FibTask& b = *new( allocate_child() ) FibTask(n-2,&y);
            set_ref_count(3); // 3 = 2 children + 1 for wait // ref_countis used to keep track of the number of tasks spawned at                            the current level of the task graph
            spawn( b );
                      // cout<<"child id of thread is \t"<<this_thread::get_id()<<"calculating n ="<<n<<endl;
            spawn_and_wait_for_all( a ); //set tasks for execution and wait for them
            *sum = x+y;
        }
        return NULL;
    }
};


long parallelFib( long n ) 
{
    long sum;
    FibTask& a = *new(task::allocate_root()) FibTask(n,&sum);
    task::spawn_root_and_wait(a);
    return sum;
}


int main()
{     
     long i,j;
     cout<<fixed;

     cout<<"Fibonacci Series parallelly formed is "<<endl;
      tick_count t0=tick_count::now();
     for(i=0;i<50;i++)
     cout<<parallelFib(i)<<"\t";
    // cout<<"parallel execution of Fibonacci series for n=10 \t"<<parallelFib(i)<<endl;

     tick_count t1=tick_count::now();
     double t=(t1-t0).seconds();
     cout<<"Time Elapsed in Parallel Execution is  \t"<<t<<endl;
     cout<<"\n Fibonacci Series Serially formed is "<<endl;
     tick_count t3=tick_count::now();

     for(j=0;j<50;j++)
     cout<<serialFib(j)<<"\t";
     tick_count t4=tick_count::now();
     double t5=(t4-t3).seconds();
     cout<<"Time Elapsed in Serial  Execution is  \t"<<t5<<endl;
     return(0);
}

Parallel Execution is taking more time as compared to serial execution.In this Parallel Execution took 2500 sec whereas serial took around 167 secs. Can anybody pls explain reason for this?

Answer 1

Overhead.

When your actual task is lightweight, the coordination/communication dominates and you do not (automatically) gain from parallel execution. This is a pretty common issue.

Try instead to compute M Fibonacci numbers (of a high enough cost) serially, then compute them in parallel. You should see a gain.

Answer 2

Change Cutoff to 12, compile with optimization on (-O on Linux; /O2 on Windows), and you should see significant speedup.

There is plenty of parallelism in the example. The problem is that with Cutoff=2, the individual units of useful parallel computation are swamped by scheduling overhead. Raising the Cutoff value should resolve the problem.

Here is the analysis. There are two important times for analyzing parallelism:

• work - the total amount of computational work.
• span - the length of the critical path.

The available parallelism is work/span.

For fib(n), when n is sufficiently large, the work is roughly proportional to fib(n) [yes, it describes itself!]. The span is the depth of the call tree - it is roughly proportional to n. So the parallelism is proportional to fib(n)/n. So even for n=10, there is plenty of available parallelism to keep a typical 2013 desktop machine humming.

The problem is that TBB tasks take time to create, execute, synchronize, and destroy. Changing Cutoff from 2 to 12 allows the serial code to take over when the work is so small that scheduling overheads would swamp it. This is a common pattern in recursive parallelism: recurse in parallel until you are down to chunks of work that might as well be done serially. In Other parallel frameworks (like OpenMP or Cilk Plus) have the same issue: there is overhead for tasks, albeit they may be more or less than TBB. All that changes is what the best threshold value is.

Try varying Cutoff. Lower values should give you more parallelism but more scheduling overhead. Higher values give you less parallelism but less scheduling overhead. In between, you will likely find a range of values that give good speedup.

Answer 3

Without more information it will be hard to tell. you need to check:How many processros your computer have? were there any other programs which might have made use of ther processors? if you want to run in (true) parallel and gain performance benefits, than the Operating system must be able to allocate at least 2 free processors. Also, for small tasks , the overhead of allocating threads and collecting their result might exceed the benefits of parallel execution.

Answer 4

Am I right in thinking that each task does result of fib(n-1) + result of fib(n-2) - so essentially, you start a task, which then starts another task and so on until we have a very large number of tasks (I got slightly lost trying to count them all - I think it's n squared). And the result of each such task is used to add up the fibonacci number.

First of all, there is no actual parallel execution here (other than perhaps two independent recursive calculations). Every task relies on the result of it's subtask, and can't really do anything in parallel. On the other hand, you are performing a whole lot of work to set up each task. Not at all surprising that you don't see any benefit)

Now, if you were to calculate the fibonacci numbers 1 .. 50 by iteration, and you started, say, one task per processor core in your system, and compared that to an iterative solution using just a single loop, I'm sure that would show a much better improvement.