Hadoop循环减速器

Question

I am trying to find a way to "loop" my reducer, for example:

for(String document: tempFrequencies.keySet())
{
if(list.get(0).equals(document))
{
testMap.put(key.toString(), DF.format(tfIDF));
}
}
//This allows me to create a hashmap which i plan to write out to context as Filename = key then all of the terms weights = value (a list I can parse out in the next job)

The code currently will run through the entire reduce and give me what I want for list.get(0) but the problem is once it is finished doing that entire reduce I need it to start again for list.get(1) etc. Any ideas on how to loop the reduce phase after it has finished?

Answer 1

Nest the for loop

for(int i = 0; i < number_of_time; i++){
//your code

}

Replace the 0 with i.

Answer 2

You can use key-tag-value technique. In mapper emit (key, 0, value) for list values and (key, 1, value) for documents (?). In reducer values will be grouped by key and tag and sorted by tag for each key. You should write your own grouping comparator (and custom partitioner). PS I am using the same techique for graph processing. I can provide sample code after weekend.