简体   繁体   English

Hadoop map-reduce输出包含奇怪的字符

[英]Hadoop map-reduce output contains weird characters

I am running a map reduce job. 我正在运行地图减少工作。 When I run it on my machine which is a single node cluster the output is as shown 当我在我的机器上运行它是一个单节点集群时,输出如图所示

hduser@nikhil-VirtualBox:/usr/local/hadoop/hadoop-1.0.4$ bin/hadoop dfs -text /user/hduser/output16/part-r-00000
0   Required Genotype column (s), Must not contain NULLS for required fields, failed, 5, 1: GENE_NAME; 2: GENE_NAME; 4: GENE_NAME; 5: GENE_NAME; 9: GENE_NAME

However when I run the same on Amazon EMR on a larger dataset, I get the following with all weird characters. 但是,当我在更大的数据集上的Amazon EMR上运行相同的操作时,我得到以下所有奇怪的字符。 What might be the reason ? 可能是什么原因?

SEQorg.apache.hadoop.io.Textorg.apache.hadoop.io.Text\00\00\00\00\00\00\968\D6\FA\E1>X(.q\8B!\ABQ\00\00-\00\00\00
1537044153\8ERequired Genotype column (s), Must not contain NULLS for required fields, failed, 1, 1: VARIANT_START_POSITION; 2: VARIANT_START_POSITION; 

The header (SEQTextText) tells you that this is a SequenceFile with a org.apache.hadoop.io.Text as key and value. 标头(SEQTextText)告诉您这是一个带有org.apache.hadoop.io.Text作为键和值的SequenceFile

So this is binary and not plain text and you can read it with a SequenceFile.Reader . 所以这是二进制而不是纯文本,您可以使用SequenceFile.Reader读取它。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM