失败：执行错误，从org.apache.hadoop.hive.ql.exec.DDLTask返回代码1。 组织/ apache / hadoop / hive / serde2 / SerDe

Question

I am processing the twitter data to hive external table, but while creating hive external table I get an error. Please look into the my code in below.

1. Added the jar file to move the hive/lib location.

2. Added to the jar file hive by using following command
   I have add the flowing jars in hive lib directory:

    ADD JAR /usr/local/hive/lib/hive-serdes-1.0-SNAPSHOT.jar; 

   Please find the external hive table:

    CREATE EXTERNAL TABLE Mytweets_raw ( id BIGINT, created_at STRING, source STRING, favorited BOOLEAN, retweet_count INT, retweeted_status STRUCT<text:STRING,tuser:STRUCT<screen_name:STRING,name:STRING>>, entities STRUCT<urls:ARRAY<STRUCT<expanded_url:STRING>>, user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>, hashtags:ARRAY<STRUCT<text:STRING>>>, text STRING, tuser STRUCT<screen_name:STRING,name:STRING,friends_count:INT,followers_count:INT,statuses_count:INT,verified:BOOLEAN,utc_offset:INT,time_zone:STRING>, in_reply_to_screen_name STRING ) ROW FORMAT SERDE 'com.cloudera.hive.serde.JSONSerDe' LOCATION 'hdfs://localhost:54310/data/tweets_raw'; 

   After this I am getting following error message, please any body can help on this?

More information: My current environment is

• Hadoop 2.9.0
• Hive 2.3.2

Answer 1

This worked for me, I just needed to replace the serde format from 'com.cloudera.hive.serde.JSONSerDe' to 'org.openx.data.jsonserde.JsonSerDe', and the error was resolved in my case. Good luck !!

 CREATE EXTERNAL TABLE tweets4 (
    id BIGINT,
    created_at STRING,
    source STRING,
    favorited BOOLEAN,
    retweet_count INT,
    retweeted_status STRUCT<
    text:STRING,
    `user`:STRUCT<screen_name:STRING,name:STRING>>,
    entities STRUCT<
    urls:ARRAY<STRUCT<expanded_url:STRING>>,
    user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>,
    hashtags:ARRAY<STRUCT<text:STRING>>>,
    text STRING,
    `user` STRUCT<
    screen_name:STRING,
    name:STRING,
    friends_count:INT,
    followers_count:INT,
    statuses_count:INT,
    verified:BOOLEAN,
    utc_offset:INT,
    time_zone:STRING>,
    in_reply_to_screen_name STRING
 )
 ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
 LOCATION '/twitter-project/project1';

Answer 2

Error code 1 signifies permission issue. It will be really difficult to identify the issue (Mostly environment issue). There can be many reasons why we are getting this error. Try below cases and see if you are getting same error.

1) Check you have permissions to create tables(Hive path).

2) Create simple Hive external table(You can confirm you have access for meta store)

3) Create Simple External Hive table by pointing to HDFS location( You can confirm your access permissions for HDFS path)

4) Create Simple External Hive table by pointing to local path ( You can confirm your access permissions for local path)

5) Check you have 777 permissions for the jar location.

I suggest try multiple cases where there is possibility of permission issue apart from the above.