Hadoop 3.3 和 oozie 5.2.0

Question

I am using hadoop 3.3 and oozie 5.2.0 .

I am getting below error:

Exception in thread "main" java.lang.NullPointerException
    at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.unregisterApplicationMaster(AMRMClientImpl.java:519)
    at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.unregisterApplicationMaster(AMRMClientAsyncImpl.java:220)
    at org.apache.oozie.action.hadoop.LauncherAM.unregisterWithRM(LauncherAM.java:354)
    at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:280)
    at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:155)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845)
    at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:143)

Answer 1

There Might be some hadoop3.3 incompatibility oozie 5.2.0

https://issues.apache.org/jira/browse/OOZIE-3219?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel&focusedCommentId=17156510#comment-17156510

Answer 2

Default Hadoop Version for Oozie 5.2 is 2.6.0

You'd likely need to build Oozie against a newer version of Hadoop

http://oozie.apache.org/docs/5.2.0/ENG_Building.html

Answer 3

There are alternatives because there is a current project call Apache Bigtop that compile Hadoop ecosystem tools with new versións. So, you have two options: the first one is use current Bigtop version 3.1.1 , who runs over oozie 5.2.1 and hadoop 3.2.4, or using the Github history changes of Bigtop as a reference to compile as your own. In the other hand, there are some blogs in chinese talking about it like this https://blog.csdn.net/tutu_ai/article/details/124372422