laravel - 在多个作业结束后执行作业

Question

I am working on a project where user gives me some input, and then I process those inputs. This processing takes long time, so I am using queues, and I want to send mail to the user that their processed data is ready on my app. What I want to achieve is, put all these jobs into the queue, then process them, and when a user's all processing ends, to mail that user.

I am using 2 phase queue. First, I get the input and dispatch a job where the input gets parsed and divided into sub-datasets, and dispatch another job from that job which processes the parsed input data from the first job. Why I did this is, the first job takes less time than the second job, and I want to use jobs to only do the processing, without any interruption or extra work with parsing, the first phase is all about parsing and the second part is all about processing. The processing part may fail and the queue will process it again if it fails, so if I do not divide the input into the sub-jobs, a fail near the end of the job may cause all the processing for the job to be done again.

If I put all those jobs to the same queue, and assign that mailing job to the Queue::after method, a user may have to wait for all the queue to finish, if I understood correctly. I want to be able to send a mail to the user when all of his inputs have been processed, without waiting any other user's inputs to be processed. The documents for Laravel Queue shows an example method:

Queue::after(function (JobProcessed $event) {
    // $event->connectionName
    // $event->job
    // $event->job->payload()
});

And what I understood from this is this works after every processed job, and I can access the job properties with the $event object. This is not going to work with my 2-phase queue structure. I know I can use some counters in database to check if all the processing is done, but I want to know if there is a better and more elegant way of achieving this.

Thank you very much.

Answer 1

It has been a few years now, but Laravel 8 may provide a solution to this:

https://github.com/laravel/framework/pull/32830

Jobs can be run in a batch, with callbacks that are invoked when all jobs complete, any jobs fail, or all jobs complete successfuly.