How to run only one job at time in same laravel queue?

Question

I have saas service which working with API. It has limits so I need that one user account doing only one request at the same time.

For this I queuing with OnQueue($user->name);

then in handle() doing job...

I need only one job can be run in users queue. At the same time may be run only diffent queues 1 job per 1 queue.

Im using redis connection.

This my job class:

 public function __construct(Accounts $acc)
    {
        $this->acc = $acc;
        $this->ownjob = $acc->prepareJobQueue();
    }

    public function handle()
    {
        $acc = $this->acc;
        $job = $this->ownjob;

        $api = new Api([
            'login'     => $acc->login,
            'password'  => $acc->password,
        ]);
        if ($api->checkLogin()) {
            info("{$acc->login} OK Authorized");
            foreach ($job['queue'] as $term) {
                switch($term['type']) {
                    case 'hashtag':
                        info("{$acc->login} Queuing: type - {$term['type']}, value - {$term['value']}");
                        $hashtag = Hashtags::where('cha_name',$term['value'])->first();
                        $answer = $api->getUsersByHashtag($hashtag,50);
                        break;
                    case 'concurency':
                        info("{$acc->login} Queuing: type - {$term['type']}, value - {$term['value']}");
                        $soc_user = Users::where('soc_unique_id',$term['value'])->first();
                        $answer = $api->getUserFollowers($soc_user);
                        break;
                    default:
                        break;
                }
            }
        } else {
            info("{$acc->login} NOT Authorized - STOP JOB");
        }

    }

This is how I dispatching job:

$accounts = Accounts::select(['id', 'login', 'hashtag_filter', 'concurency_filter'])->whereNotNull('hashtag_filter')->get();
        foreach ($accounts as $acc) {
            doFollowing::dispatch($acc)->onQueue($acc->login);
        }

Answer 1

You could use Laravel's builtin rate limiting for this (note that it does require Redis).

Inside your job:

Redis::funnel('process-name')->limit(1)->then(function () {
    // Your job logic here
});

Note that if you don't provide a second callback to then, it will throw an exception if it cannot obtain the lock (which will cause your job to fail)

If you're using Laravel 6 or higher you could also opt to do this in a job middleware, rather than in the job itself (handy if you have multiple jobs that share the same lock)

More info on Laravel's rate limiting: https://laravel.com/docs/5.8/queues#rate-limiting

Answer 2

You could limit numprocs per queue in your Supervisor or Horizon setup.

If you only spawn one queue worker per user I believe you will get your desired behaviour.

Answer 3

Use mxl/laravel-queue-rate-limit Composer package.

It enables you to rate limit Laravel jobs on specific queue without using A third party driver such as Redis .

1. Install it with:

    $ composer require mxl/laravel-queue-rate-limit:^1.0

2. This package is compatible with Laravel 5.5+ and uses [auto-discovery][1] feature to add MichaelLedin\\LaravelQueueRateLimit\\QueueServiceProvider::class to providers.

3. Add rate limit (x number of jobs per y seconds) settings to config/queue.php :

    'rateLimit' => [ 'mail' => [ 'allows' => 1, // number of jobs 'every' => 5 // time interval in seconds ] ]

   These settings allow to run 1 job every 5 seconds on mail queue. Make sure that default queue driver ( default property in config/queue.php ) is set to any value except sync .

4. Run queue worker with --queue mail option:

    $ php artisan queue:work --queue mail

   You can run worker on multiple queues, but only queues referenced in rateLimit setting will be rate limited:

    $ php artisan queue:work --queue mail,default

   Jobs on default queue will be executed without rate limiting.

5. Queue some jobs to test rate limiting:

    SomeJob::dispatch()->onQueue('mail'); SomeJob::dispatch()->onQueue('mail'); SomeJob::dispatch()->onQueue('mail'); SomeJob::dispatch();