How can I retrieve the execution status of parallel triggered child jobs to a pipeline script

Question

have a pipeline script that executes child jobs in parallel.

Say I have 5 data (a,b,c,d,e) that has to be executed on 3 jobs (J1, J2, J3)

My pipeline script is in the below format

for (int i = 0; i < size; i++) { def index = i branches["branch${i}"] = { build job: 'SampleJob', parameters: [ string(name: 'param1', value:'${data}'), string(name:'dummy', value: "${index}")] } } parallel branches

My problem is, say the execution is happening on Job 1 with the data 1,2,3,4,5 and if the data 3 execution is failed on Job 1 then the data 3 execution should be stopped there itself and should not happen on the subsequent parallel execution on Jobs 2 and 3.

Is there any way that I can read the execution status of parallelly execution job status on the Pipeline script so that I can restrict data 3 execution to block in Jobs 2 and 3.

I am quite blocked here for a long time. Hoping for a solution from my community. Thanks a lot in advance.

Answer 1

In summary, it sounds like you want to

• run multiple jobs in parallel against different pieces of data. I will call the set of related jobs the "batch".
• avoid starting a queued job if any of the jobs in the batch have failed
• automatically abort a running job if any of the jobs in the batch have failed

The jobs need some way to communicate their failure to the others. Use a shared storage location to store the "failure flag". If the file exists, then one or more of the jobs have failed. For example, a shared NFS path: /shared/jenkins/jobstate/<BATCH_ID>/failed

At the start of the job, check for the existence of this path. Exit if it does. The file doesn't necessarily need to contain any data - its presence is enough.

Since you need running jobs to abort early if the failure flag exists, you will need to poll that location periodically. For example, after each unit of work. Again, if the file exists then exit early.

If you don't use NFS, that's ok. You could also use an object storage bucket. The important thing is that the state is accessible to all the relevant build jobs.