简体   繁体   English

Hadoop Mapreduce中的奇怪输出

[英]Strange output in Hadoop mapreduce

This is a sample from the input file: 这是来自输入文件的示例:

1,name1,name2 
2,name3,name4 
3,name5,name6

and this is my map method: 这是我的地图方法:

public void map(LongWritable key, Text value, OutputCollector<Text, Text> output, Reporter reporter) throws IOException 
{
    String line = value.toString();
    StringTokenizer tk = new StringTokenizer( line, ",");       
    String keyValue = tk.nextToken();
    String s1Value = tk.nextToken();
    String s2Value = tk.nextToken();
    String valueString = s1Value+","+s2Value;
    output.collect( new Text(keyValue), new Text(valueString) );
}

and this is my reduce function: 这是我的reduce函数:

public static class Reduce extends MapReduceBase implements Reducer<Text, Text, Text, Text> 
{
    public void reduce(Text key, Iterator<Text> values, 
        OutputCollector<Text, Text> output, Reporter reporter) throws IOException 
    {
        String item="";
        Text tmp= new Text();
        while ( values.hasNext() ) 
        {
            tmp = values.next();
        }
        item = tmp.toString();

        StringTokenizer tk = new StringTokenizer( item, ",");

        String s1="";
        String s2="";
        boolean entered = false;
        try
        {
            while ( tk.hasMoreTokens() && !entered )
            {   
                s1 = tk.nextToken();
                s2 = tk.nextToken();
                entered = true;
            }
        }
        catch (Exception e )
        {
            System.out.println("PROBLEM:"+item);
        }
        double result = compare(s1,s2);
        String result2 = s1+" & "+s2+"="+result;
        output.collect( key, new Text(result2) );
    }
}

So i expect the output to be (eg): 所以我希望输出是(例如):

name1 & name2=1.0  

But what I get is: 但是我得到的是:

name1 & name2=1.0  &  =0.0

looks like all the time there are two empty strings get compared!! 看起来一直都有两个空字符串被比较!! Why there are always empty strings? 为什么总是有空字符串?

It should care the code of "compare(s1,s2)"; 它应该注意“ compare(s1,s2)”的代码; Please paste the code of compare function. 请粘贴比较功能的代码。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM