将Apache Pig连接到Hadoop集群

Question

I'm using Apache Pig to do some data analysis work with a Hadoop cluster. I deployed ONE muster node and 32 slave nodes in the hadoop cluster. However, when I use Pig to run scripts in mapreduce mode, connecting to that Hadoop cluster, it always initiate only one map and one reduce. How can I set up Pig or Hadoop to make use of all 32 slaves?

Job status is shown below:

Job Stats (time in seconds):
JobId   Maps    Reduces MaxMapTime  MinMapTime  AvgMapTime  MedianMapTime   MaxReduceTime   MinReduceTime   AvgReduceTime   MedianReducetime    Alias   Feature Outputs
job_1457865367374_0001  1   1   88  88  88  88  27  27  27  27  1-1,access_grouped,access_summed,cleaned,named,raw,timed,timed_grouped,timed_summedMULTI_QUERY  
job_1457865367374_0002  1   1   5   5   5   5   5   5   5   5   access_ordered  SAMPLER 
job_1457865367374_0003  2   1   10  10  10  10  6   6   6   6   density,density_scored  HASH_JOIN   
job_1457865367374_0004  1   1   5   5   5   5   5   5   5   5   timed_ordered   SAMPLER 
job_1457865367374_0005  1   1   5   5   5   5   5   5   5   5   timed_ordered   ORDER_BY    hdfs://master:54310/user/ubuntu/Data/timed_ordered,
job_1457865367374_0006  1   1   5   5   5   5   5   5   5   5   access_ordered  ORDER_BY    hdfs://master:54310/user/ubuntu/Data/access_ordered,
job_1457865367374_0007  1   1   5   5   5   5   5   5   5   5   density_ordered SAMPLER 
job_1457865367374_0008  1   1   5   5   5   5   5   5   5   5   density_ordered ORDER_BY    hdfs://master:54310/user/ubuntu/Data/density_ordered,

btw, I installed Apache Pig on the master machine.

Answer 1

SET default_parallel xyz

In pig above command gives power to set no. of parallel tasks.but hadoop Framework detects no. of mapper(based on no. of input splits) and reducers(can be set on custer level or application level.). You cannot set no. of mappers for your appication but reducers, you can.