[英]Expected BEGIN_OBJECT but was STRING at line 1 column 6
My java code 我的java代码
public class Recipe {
public static class TokenizerMapper extends Mapper<Object, Text, Text, IntWritable>{
private final static IntWritable one = new IntWritable(1);
private Text word = new Text();
Gson gson = new Gson();
public void map(Object key, Text value, Context context ) throws IOException, InterruptedException {
Roo roo=gson.fromJson(value.toString(), Roo.class);
if (roo.manner_of_death != null) {
word.set(roo.manner_of_death);
} else {
word.set("none");
}
context.write(word, one);
}
}
public static class IntSumReducer
extends Reducer<Text,IntWritable,Text,IntWritable> {
private IntWritable result = new IntWritable();
public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException {
int sum = 0;
for (IntWritable val : values) {
sum += val.get();
}
result.set(sum);
context.write(key, result);
}
}
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
String[] otherArgs = new GenericOptionsParser(conf, args).getRemainingArgs();
if (otherArgs.length != 2) {
System.err.println("Usage: recipe <in> <out>");
System.exit(2);
}
@SuppressWarnings("deprecation")
Job job = new Job(conf, "Recipe");
job.setJarByClass(Recipe.class);
job.setMapperClass(TokenizerMapper.class);
job.setCombinerClass(IntSumReducer.class);
job.setReducerClass(IntSumReducer.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
FileInputFormat.addInputPath(job, new Path(otherArgs[0]));
FileOutputFormat.setOutputPath(job, new Path(otherArgs[1]));
// FileInputFormat.addInputPath(job, new Path("hdfs://127.0.0.1:9000/in"));
// FileOutputFormat.setOutputPath(job, new Path("hdfs://127.0.0.1:9000/out"));
System.exit(job.waitForCompletion(true) ? 0 : 1);
// job.submit();
}
}
class Id
{
public String oid;
}
class Roo
{
public Id _id ;
public String resident_status;
public String month_of_death;
public String sex;
public String marital_status;
public String manner_of_death;
public String autopsy;
public String race;
}
My JSON 我的JSON
{
"_id" : ObjectId("5bfc49155fa79a44dca1f9b9"),
"resident_status" : "1",
"month_of_death" : "06",
"sex" : "M",
"marital_status" : "M",
"manner_of_death" : "7",
"autopsy" : "N",
"race" : "02"
}
{
"_id" : ObjectId("5bfc49155fa79a44dca1f56c"),
"resident_status" : "1",
"month_of_death" : "03",
"sex" : "F",
"marital_status" : "D",
"manner_of_death" : "7",
"autopsy" : "N",
"race" : "01"
}
All fields are string except id 除ID外,所有字段均为字符串
My error 我的错误
18/11/26 18:02:55 INFO mapreduce.Job: Task Id : attempt_1543189350698_0010_m_000000_0, Status : FAILED Error: com.google.gson.JsonSyntaxException: java.io.EOFException: End of input at line 1 column 3 at com.google.gson.Gson.fromJson(Gson.java:813) 18/11/26 18:02:55 INFO mapreduce.Job:任务ID:try_1543189350698_0010_m_000000_0,状态:FAILED错误:com.google.gson.JsonSyntaxException:java.io.EOFException:com的第1行第3列的输入结束。 google.gson.Gson.fromJson(Gson.java:813)
18/11/26 18:02:55 INFO mapreduce.Job: Task Id : attempt_1543189350698_0010_m_000001_0, Status : FAILED Error: com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 18/11/26 18:02:55 INFO mapreduce.Job:任务ID:try_1543189350698_0010_m_000001_0,状态:FAILED错误:com.google.gson.JsonSyntaxException:java.lang.IllegalStateException:预期为BEGIN_OBJECT,但在第1行STRING
When you do this, 当您这样做时
class Id
{
public String oid;
}
class Roo
{
public Id _id ;
You're telling Gson that it is trying to parse this type of object 您告诉Gson它正在尝试解析此类对象
{
"_id" : {
"oid" : "5bfc49155fa79a44dca1f56c"
},
...
}
Which is not what you have, and you don't have valid JSON anyway because ObjectId
doesn't have quotes around it. 这不是您所拥有的,而且您也没有有效的JSON,因为ObjectId
周围没有引号。
Second problem - MapReduce defaults to reading single lines of data, which would be fine if your input file was only two lines like so, but the JSON still should be valid 第二个问题-MapReduce默认读取单行数据,如果您的输入文件只有两行,这很好,但是JSON仍然应该有效
{ "_id" : ... }
{ "_id" : ... }
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.