shell 脚本中的 Hive SQL 在 oozie 中失败

Question

i have an hive HQL that needs to be called in a Shell script. it works fine when I execute the .Sh file in linux. But when I execute the same .sh in oozie, the hive part doesn't seem to work. I am unable to see the error in yarn.

.sh file is as below

export UDR_START_DT=default.udr_lt_bc_$(date +"%Y%m%d" -d "yesterday")

echo "Start date : "$UDR_START_DT

hive -f /home/scripts/hive/tab_create.hql

---

the tab_create.hql is a below:

CREATE TABLE default.san_tst
(
alt_rated_amount double ,
alt_rated_currency double ,
alt_tariff_clicks_volume double ,
alt_tmcode double ,
bill_display_ind string
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ",";

note: the path of the .hql file is in linux and its not the HDFS path.

Answer 1

Oozie takes a copy of this shell script and executes in any one of the node in your cluster. So the hive path you are referring to might not be present in that node.

If it is single node cluster, make sure oozie user has access to that complete path or copy the hql file into your oozie workflow directory in hdfs.

Add file into your shell action:

<file>oozie_workflow_dir/scripts/hive/tab_create.hql#tab_create.hql</file>

However, you dont need to use shell script to execute hql files. Hive action is better and suggested to run hive scripts.