In Hadoop mapreduce does all mappers need to communicate with all reducers?

Question

In mapreduce i know that mappers dont communicate with each other. But does all mappers need to communicate with all reducers.?

Answer 1

It depends on the set of keys that the mapper produces. If it doesn't produce any keys that need to be handled by a particular reducer, it may not need to communicate with that reducer at all. But the keys produced by a mapper typically depend on its input, and the input can vary from job to job, so it's possible (and even likely) that for some jobs, every mapper will need to send data to every reducer.