简体   繁体   中英

Hive: Invoking shell commands from hive-script throws error

I have a hive script that executes some DMLs and drop some tables, and executes some shell-delete files. I am firing the script using hive -f myscript.hql .
From within the script I need to remove files from local directory. I tried to use !rm /home/myuser/temp_table_id_*; throws error:

rm: cannot remove ‘/home/myuser/temp_table_id_*’: No such file or directory
Command failed with exit code = 1

* is not working.
Here is a sample script:

--My HQL File--
INSERT OVERWRITE ....
...
..;
DROP TABLE TEMP_TABLE;
!hadoop fs -rm -r /user/myuser/ext_tables/temp_table;
!rm /home/myuser/temp_table_id_*;

CREATE TABLE NEW_TABLE(
....
...
;

I am calling the script with the command: hive -f myscript.hql The script is running fine till it finds the line : !rm /home/myuser/temp_table_id_*; where is cursing about the * .
When I am providing separate file names instead of the * , its working.
But i wish to use * .

Try

dfs -rm /home/myuser/temp_table_id_*;

in the HQL. Wildcard works well with hive dfs commands.

From Hive docs

dfs <dfs command> -Executes a dfs command from the Hive shell.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM