映射器可以写入多个文件吗

Question

I am new to Hadoop and Map reduce and I am using an old version of hadoop 0.19. I have a program that reads a file/excel and gives me the column contents as a list of places,location,names etc.

Lets assume I have the mapper dividing my input file into 2 parts. Each of these mappers will give me a list of the above mentioned entities.

My question is:

1. How do I maintain track of data and save list of places and names separately for each file from each mapper.How will reducer recognize these files and come up with consolidated list of places and another of names for each file.

Say Doc-1:

list of places from mapper1---NY,1 US,2
list of names from mapper1---James 3 ,Ron 8
list of places from mapper-2 --NY 6 UK 5
list of names from mapper 2--Kate 9

Something like this.

How do I save the output from each mapper and for each type of entity as in name or place.

How will reducer recognize and reduce only names and come up with a final list or only locations and come up with a final list pertaining to that file.

Pls help me with this and let me know any methods that help me do that in Java.

Answer 1

If this is a Map-only job, there will be the same number of output files as there are Mappers. If this is a MapReduce job, you can specify the number of Reducers. Provide a Partitioner that sends the data from a specific Mapper to a specific Reducer. If you are not sure of the number of Mappers, make the number of Reducers slightly higher that total number and only use the first n Reducers from Partitioner.