无法通过oozie运行shell脚本

Question

I'm trying to run a shell script that just create a directory via mkdir command via oozie workflow.

I'm using HDP 2.6.5 | Oozie 4.2.0

The error message is always :

java.io.IOException: Cannot run program "test.sh" (in directory "/hadoop/yarn/local/usercache/whorchani/appcache/application_1547225966242_3390/container_e111_1547225966242_3390_01_000002"): error=2, No such file or directory

Answer 1

It's hard to answer your question if you don't at least post your Oozie XML for the relevant action. My guess is that you haven't used the <file> tag.

From Apache Oozie by Mohammad Kamrul Islam and Aravind Srinivasan

Because the shell command runs on any Hadoop node, you need to be aware of the path of the binary on these nodes. The executable has to be either available on the node or copied by the action via the distributed cache using the <file> tag. For the binaries on the node that are not copied via the cache, it's perhaps safer and easier to debug if you always use an absolute path.

Here is a simple example of a shell action:

    <action name="shell_action">
        <shell xmlns = "uri:oozie:shell-action:0.1">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <exec>sh</exec>
            <argument>my_shell_script.sh</argument>
            <file>/full/hdfs/path/to/your/script/my_shell_script.sh</file>
        </shell>
        <ok to="action2"/>
        <error to="fail"/>
    </action>