Hadoop的slowtaskthreshold是否有助于解决竞争状况？

Question

I had a question about mapreduce.job.speculative.slowtaskthreshold .

The docs say:

The number of standard deviations by which a task's ave progress-rates must be lower than the average of all running tasks' for the task to be considered too slow.

I'm curious what happens when a process is considered "too slow". Does it kill and restart it? Just kill? I'm curious because I think I've possibly encountered a race condition and would like to tune the parameters to minimize the issue.

Source: http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml

Answer 1

With speculative execution, when the framework decides that a task is "too slow", it will speculatively execute the same task on a different node. Once one of the two tasks finishes successfully, the other task is killed.