Unable to Pass option using getopts to oozie shell Action

Question

I created a script in shell and passing the arguments using getopts methods in my script like this:

sh my_code.sh -F"file_name"

where my_code.sh is my unix script name and file_name is the file I am passing to my script using getopts.

This is working fine when I am invoking my script from the command line.
I want to invoke the same script by using oozie, but I am not sure how can I do it.

I tried passing the argument to the "exec" as well as "file" tag in the xml

When I am trying passing argument in exec tag, it was giving "JavaNullPoint" Expection

exec TAG

<exec>my_code.sh -F file_name</exec>

file TAG

<file>$/user/oozie/my_code.sh#$my_code.sh -F file_name</file>

When I am trying passing argument in File Tag, I was getting error, "No such File or directory". It was searching the file_name in /yarn/hadoop directory.

Can anyone please suggest how can I achieve this by using oozie?

Answer 1

You need to create a lib/ folder as part of your workflow where Oozie will upload the script as part of its process. This directory should also be uploaded to the oozie.wf.application.path location.

The reason this is required is that Oozie will run on any random YARN node, and pretend that you had a hundred node cluster, and you would otherwise have to ensure that every single server had /user/oozie/my_code.sh file available (which of course is hard to track). When this file can be placed on HDFS, every node can download it locally.

So if you put the script in the lib directory next to the workflow xml that you submit, then you can reference the script by name directly rather than using the # syntax

Then, you'll want to use the argument xml tags for the opts

https://oozie.apache.org/docs/4.3.1/DG_ShellActionExtension.html

Answer 2

I have created lib/ folder and uploaded it to oozie.wf.application.path location.

I am able to pass files to my shell action.