i took example from cloudera website to write a custom SerDe for parsing a file
http://blog.cloudera.com/blog/2012/12/how-to-use-a-serde-in-apache-hive/
it seems a good example but when i create table with custom serde
ADD JAR <path-to-hive-serdes-jar>;
CREATE EXTERNAL TABLE tweets (
id BIGINT,
created_at STRING,
source STRING,
favorited BOOLEAN,
retweeted_status STRUCT<
text:STRING,
user:STRUCT<screen_name:STRING,name:STRING>,
retweet_count:INT>,
entities STRUCT<
urls:ARRAY<STRUCT<expanded_url:STRING>>,
user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>,
hashtags:ARRAY<STRUCT<text:STRING>>>,
text STRING,
user STRUCT<
screen_name:STRING,
name:STRING,
friends_count:INT,
followers_count:INT,
statuses_count:INT,
verified:BOOLEAN,
utc_offset:INT,
time_zone:STRING>,
in_reply_to_screen_name STRING
)
PARTITIONED BY (datehour INT)
ROW FORMAT SERDE 'com.cloudera.hive.serde.JSONSerDe'
LOCATION '/user/flume/tweets';
it executed perfectly fine but when i do
select * from tweets;
i am getting nothing so thats why i wanted to know if i can run hive in debug mode to see where it is getting failed
你最好通过将logger模式切换到DEBUG来启动hive shell,如下所示,我希望你能从那里找到有用的东西。
hive --hiveconf hive.root.logger=DEBUG,console
可以调试Hive代码。这个链接可以帮助您: https : //cwiki.apache.org/confluence/display/Hive/DeveloperGuide#DeveloperGuide-DebuggingHiveCode
Setting hive --hiveconf hive.root.logger=DEBUG,console
may not always work because of company specific setup.
I ended up creating a hive-log4j.properties
file in my home directory with following settings:
log4j.rootCategory=DEBUG,console
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.target=System.err
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n
and started hive shell using CLASSPATH=$HOME hive
which adds your home directory having hive-log4j.properties
in front of the classpath and so is picked up.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.