Unable to run shell script using oozie shell action

Question

I created a oozie workflow for shell action. Below is the simple oozie shell action. I put the Shell script in workflow lib folder. This shell script is doing below things:

1) Fetching encrypted json from s3 bucket and putting them into local folder

2) Decrypting json and putting it in one hdfs location.

3) For aws connection and decrypting json, its using some keys ie pem and pub keys.

4) I put those keys in lib folder along with the shell script.

<workflow-app xmlns="uri:oozie:workflow:0.5" name="LaborIngestion">
<start to='LaborIngestion'/>
    <action name='LaborIngestion'>
        <shell xmlns="uri:oozie:shell-action:0.1">
                <job-tracker>${jobTracker}</job-tracker>
                <name-node>${nameNode}</name-node>
                <exec>pullDataS3.sh</exec>
        </shell>

        <ok to="end"/>
        <error to="fail"/>
    </action>

    <kill name="fail">
        <message>LaborHrExtract workflow failed, error message[${wf:errorMessage(wf:lastErrorNode())}]
        </message>
    </kill>
    <end name="end"/>
</workflow-app>

The problem I am facing is that this workflow ends within 7-8 seconds with success status but its fetching any file. Usually when I run this script from command prompt, it takes around 2-3 minutes. I am not sure if I am putting decryption keys and connection keys properly. Please suggest me if this is the right way to put keys in oozie.

Moreover I ran one simple script with below hadoop command with the same workflow and it worked fine.

hadoop fs -mkdir /com/kapil/kumar

Answer 1

In your workflow after

<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<exec>pullDataS3.sh</exec>

Yow will need to provide the HDFS path of the location of the shell script.

Then It should work. Something like below

<file>/user/xxxxx/oozie/import/lib/pullDataS3.sh#pullDataS3.sh</file>