[英]Compress Json data in hive external table, at the time querying throwing exception?
I have created external tables by following below steps 我按照以下步骤创建了外部表
Hive > ADD JAR /usr/lib/hive/lib/hive-serdes-1.0-SNAPSHOT.jar;
Hive > set hive.exec.compress.output=true;
Hive > set mapred.output.compress=true;
Hive> set mapred.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec;
Hive> set io.compression.codecs=org.apache.hadoop.io.compress.GzipCodec;
Hive > CREATE EXTERNAL TABLE Json (id BIGINT,created_at STRING,source STRING,favorited BOOLEAN) ROW FORMAT SERDE "com.cloudera.hive.serde.JSONSerDe"
LOCATION /user/cloudera/ jsonGZ ";
I have compressed my Json file by executing below command 我通过执行以下命令压缩了我的Json文件
“ hadoop jar /usr/lib/hadoop-mapreduce/hadoop-streaming-2.6.0-cdh5.5.0.jar -Dmap.output.compress=true -Dmap.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec -Dmapreduce.output.fileoutputformat.compress=true -Dmapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.GzipCodec -input /user/cloudera/json/ -output /user/cloudera/jsonGZ “
Then when I am running “ select * from json; “
然后,当我正在运行“ select * from json; “
“ select * from json; “
I am getting the below error: “ select * from json; “
我收到以下错误:
“OK Failed with exception java.io.IOException:org.apache.hadoop.hive.serde2.SerDeException: org.codehaus.jackson.map.JsonMappingException: Can not deserialize instance of java.util.LinkedHashMap out of VALUE_NUMBER_INT token at “
And also I have created one more table using “org.apache.hive.hcatalog.data.JsonSerD” 而且我还使用“org.apache.hive.hcatalog.data.JsonSerD”创建了另一个表
Hive > ADD JAR /usr/lib/hive-hactalog/share/ hactalog/ hive-hactalog-core.jar;
Hive > CREATE EXTERNAL TABLE Json 1(id BIGINT,created_at STRING,source STRING,favorited BOOLEAN) ROW FORMAT SERDE "com.cloudera.hive.serde.JSONSerDe"
LOCATION /user/cloudera/ jsonGZ ";
Then when I am running “select * from json1;“
,I am getting the below error: 然后当我运行“select * from json1;“
,我收到以下错误:
Failed with exception java.io.IOException:org.apache.hadoop.hive.serde2.SerDeException: java.io.IOException: Start token not found where expected" after using "org.apache.hive.hcatalog.core(hive-hcatalog-core-0.13.0.jar)"
Am I missing something? 我错过了什么吗? How can I resolve this errors. 我该如何解决这个错误。
Just gzip your files and put them as is (*.gz) into the table location. 只需将您的文件gzip并按原样(* .gz)放入表格位置即可。
gzip filename gzip文件名
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.