Mapper组合器修补程序排序/排序

Question

I have the below text in Definite Guide: Hadoop in pg 206.

Before it writes to disk, the thread first divides the data into partitions corresponding to the reducers that they will ultimately be sent to. Within each partition, the background thread performs an in-memory sort by key, and if there is a combiner function, it is run on the output of the sort. Running the combiner function makes for a more compact map output, so there is less data to write to local disk and to transfer to the reducer.

So with this understanding, Can I sort the order as Mapper, partitioner, shuffle/sort, Combiner?

Answer 1

I've written a good article about this: http://0x0fff.com/hadoop-mapreduce-comprehensive-description/ In general you are right, but in particular there are much more corner cases - combiner might be omitted for some of the records, for some of them it might run many times, and it is even so that combiner might be started on reduce side before the reducer. So you are right in general, but the things are much more complex