In oozie workflow (HUE), how to pass parameter from shell action to HDFS fs action

Question

In My workflow, I have one shell action and an HDFS fs action

1. Shell action echos date. (date=2016-10-06)

2. I want to set the above date parameter in HDFS fs action mkdir path. Following is the action definition.

    <action name="fs-a347"> <fs> <mkdir path='${nameNode}/user/kylin/${wf:actionData("shell-e424")["date"]}'/> </fs> <ok to="End"/> <error to="Kill"/> </action>

I get the following error.

EL_ERROR    Encountered "&", expected one of [<INTEGER_LITERAL>, <FLOATING_POINT_LITERAL>, <STRING_LITERAL>, "true", "false", "null", "(", ")", "-", "not", "!", "empty", <IDENTIFIER>]

Answer 1

You need to concate the two strings to form the complete path.

String concat(String s1, String s2)

It returns the concatenation of 2 strings. A string with null value is considered as an empty string.

<mkdir path='concat(${nameNode}/user/kylin/, ${wf:actionData("shell-e424")["date"]})'/>

Answer 2

In order to capture the output from your script or any Oozie action node, you need to utilize the <capture-output/> tag at the end of your node definition.

From the Oozie documentation:

If the capture-output element is present, it indicates Oozie to capture output >of the STDOUT of the shell command execution. The Shell command output must be >in Java Properties file format and it must not exceed 2KB. From within the >workflow definition, the output of an Shell action node is accessible via the >String action:output(String node, String key) function (Refer to section '4.2.6 >Action EL Functions').

The syntax and full specifications can be found on the Oozie documentation for Shell actions here:http://oozie.apache.org/docs/4.0.0/DG_ShellActionExtension.html

Answer 3

I ran into the same situation, and it's most likely to do with the vagaries of the XML parsing interacting with the wf:actionData() EL function. Instead of using double quotes for the map, swap to single quotes, like so:

<action name="fs-a347">
    <fs>
        <mkdir path="${nameNode}/user/kylin/${wf:actionData('shell-e424')['date']}"/>
    </fs>
    <ok to="End"/>
    <error to="Kill"/>
</action>