在Spark中执行mapreduces

Question

My teacher said that we can execute MapReduces in Spark. But, since Spark is said to be faster that hadoop, this means that is always better to do mapreduces in spark. So, Hadoop MapReduces become useless. Is this correct ?

Answer 1

You can execute map() and reduce() function operations on Spark RDD and DataFrames.

I think this is what your teacher meant

Also Spark is not faster than Hadoop - it's complimentary to Hadoop, it might be faster than Mapreduce, but given the proper resource allocation, Tez execution can actually be faster than Spark and MapReduce and require less total resources than Spark. Unfortunately, one doesn't just write Tez code in most cases, it's usually actions in Pig or Hive