How to get list of yarn finished/failed jobs of yesterday only

Question

I tried with below command

$ yarn application -list -appState FINISHED

It is giving me list of all the finished jobs but my purpose is to last finished jobs for yesterday date. Also the list doesn't not give the date in the details.

Then i tried curl command

'''curl -k -X GET “http:rm:8088/cluster/apps?startedTimeBegin=122&startedTimeEnd=124” (122 & 125 is epoc time example) '''

It is also not throwing the specific details.

Im actually shell script to get list of yarn finished and failed jobs from yesterday.

Answer 1

This is really just a yarn question, not a MapR or Cloudera question.

You might have the best luck asking on the Apache Yarn user email list.

Answer 2

You can do that with combination of

yarn application -list and yarn application -status

where base of the command would look something like

for i in ``yarn application -list -appStates FINISHED | grep "^app"| cut -f1``; do yarn application -status $i; done

I will leave the parsing of the output to you. Important thing to say is that you can list at most 10k most recent apps (by default).