On Spark-cluster.Is there a parameter that controls the minimum run time of the spark job

Question

My Spark program will first determine whether the input data path exists and,if it does not,exit safely.But after exiting,yarn will retry the job once.So,I guess one parameter will control the minimum run time of the job. On Spark-cluster.Is there a parameter that controls the minimum run time of the spark job,which is to trigger a retry even if the task succeeds but is less than that time.

---------after the first edit--------------

I turned the number of retries to 1,and now I don't have to think about the number of retries. There is only one sentence System.out.println('MyProgram'); in the main method in my program.The log shows that everything is fine,but yarn thinks it's a failed job.I'm very confused.

Answer 1

否。如果您的作业以零以外的退出状态结束，则会发生重试。