Laravel队列-限制同时执行的任务数

Question

I have written a Laravel task which reads data from db, calls a shell script to process data, and that saves results to db. Tasks are queued very fast, and multiple tasks always exist in the queue. I noticed that tasks are not performed one by one, but instead couple of tasks are executed simultaneously. This is good principle, generally speaking, but it seems that shell script I call(which is btw third-party and I cannot modify it) is not thread-safe and it creates problems when called simultaneously by multiple tasks. When I am executing task function synchronously, as part of my main process, everything works smooth.

What I would like to know is if it is possible to tell Laravel queue process one task at a time, because I still want to split this job from my main processing, but also have to avoid these errors. I am on Laravel 5, using default 'sync' queue method.

Answer 1

About the tasks being executed simultaneously, have you tried adding withoutOverlapping ? http://laravel.com/docs/5.1/scheduling#preventing-task-overlaps

If you're using a closure call, you'd also need to add a name

For example:

$schedule->call(function () {
    DB::table('users')->insert([
        'user_id'       => '3',
        'name'          => 'anton'
    ]);
})->everyHour()
  ->name('nyanpasu')
  ->withoutOverlapping();

Answer 2

I couldn't find any answer that could fit my needs and this question is in top 10 search results for "laravel limit one job", so I decided that I'll share my solution to this problem. maybe it'll help someone else.

I had jobs overlapping for an API requests which resulted in 500 error on the external server. To overcome this issue I wanted to limit the execution to one job.

The simplest solution for me was to create another Supervisor configuration file for one specific queue - in my case "api" - which I limited in the config to one proces with numprocs=1 command.

After this I can dispatch all API jobs for external servers to queue "api" and they never overlap.

I don't know if it's the best approach but it fixed my problems with minimum effort.