Benchmarking Hadoop Map-Reduce application

Question

I need to compare two Hadoop scheduling algorithms by job execution time. What could I use to get the duration of the execution for all tasks?

Answer 1

You can see detailed information about the tasks and jobs from the address:

http://hostnameofmachinerunningtheJobTracker:50030/jobtracker.jsp

you can have other information from link

Answer 2

JobTracker web UI gives you very useful reports which allow to compare everything up to available logs for every mapper and reducer.

Also look on mrbench class inside hadoop-test.jar archive. There is plenty of information on the net about its usage for Hadoop cluster benchmarking like this article .