Laravel Cron Scheduler job not running as expected

Question

I have Laravel cron issue ,In Console Kernel I have defined job which will hit Rollovercron.php file every 10 mins and every time it will hit it will pass one country. atleast 100 Countries are defined in an array and will be passed one by one to Rollovercron.php according to foreach loop. Rollovercron.php file takes minimum 2 hrs to run for single country.

I have multiple issues with this cron job:

1. 100 elements in an array not getting fetched one by one means I can see 'GH' country(Ghana) has run for 5 times continuously and many of the countries are skipped.

2. when ever I get country missing issue I do composer update and clear cache frequently.

I want my cron should run smoothly and fetch all countries not even single country should miss and I should not need to do composer update for this all the time.

Please help me in this ,struggling for this since many months.

bellow is Kernel.php file:

<?php

namespace App\Console;

use Illuminate\Console\Scheduling\Schedule;

use Illuminate\Foundation\Console\Kernel as ConsoleKernel;

use DB; 

class Kernel

extends ConsoleKernel

{  


/**

* The Artisan commands provided by your application.

*

* @var array

*/

protected $commands = [

\App\Console\Commands\preAlert::class,

\App\Console\Commands\blCron::class,

\App\Console\Commands\mainRollover::class,

\App\Console\Commands\refilingSync::class,

\App\Console\Commands\TestCommand::class,

\App\Console\Commands\rollOverCron::class,

\App\Console\Commands\FrontPageRedis::class,

\App\Console\Commands\filingStatusRejectionQueue::class,

\App\Console\Commands\VesselDashboardRedis::class,

\App\Console\Commands\Bookingcountupdate::class,

//   \App\Console\Commands\Voyagetwovisit::class,

];

/**

* Define the application's command schedule.

*

* @param  \Illuminate\Console\Scheduling\Schedule  $schedule

* @return void

*/

protected function schedule(Schedule $schedule)

{


$countrylist=array('NL','AR','CL','EC','DE','PH','ID','TT','JM','KR','BE','VN','US','BR','CM','MG','ZA','MU','RU','DO','GT','HN','SV', 'PR','SN', 'TN', 'SI','CI','CR','GM','GN','GY','HR','LC','LR','MR','UY','KH','BD','TH','JP','MM','AT','IE','CH','LB','PY','KE','YT','TZ','MZ','NA','GQ','ME');


foreach ($countrylist as $country) { 

$schedule->command('rollOverCron:send ' . $country)

->everyTenMinutes()

->withoutOverlapping();



}

foreach ($countrylist as $country) { 

$schedule->command('mainRollover:send ' . $country)

->daily()

->withoutOverlapping();



}

$schedule->command('filingStatusRejectionQueue')

->hourly()

->withoutOverlapping();





$schedule->command('Bookingcountupdate')

->everyTenMinutes()

->withoutOverlapping();






$schedule->command('preAlert')

->hourly()

->withoutOverlapping();




}
protected function commands()

{

require base_path('routes/console.php');

}

}



/**

* Register the Closure based commands for the application.

*
* @return void

*/

Answer 1

Laravel scheduling, knowing how it works helps, so you can debug it when it doesn't work as expected. This does involve diving in the source.

You invoke command on the scheduler, this returns an event .
Let's check how Laravel decides what defines overlapping , we see it expires after 1440 minutes, aka 24 hours .
So after one day, if the scheduled items have not run these scheduled items just stop being scheduled. We see that a mutex is being used here. Let's see where it comes from. It seems it's provided in the constructor .
So lets see which mutex is being provided. In the exec and the call functions the mutex defined in the Scheduler constructor is used. The mutex used there is an interface, probably used as a Facade, and the real implementation is most likely in CacheSchedulingMutex , which creates a mutex id using the mutexName from the event and the current time in hours and minutes.
Looking at the mutexName we see that the id exists out of the expression and command combined.

To summarise, all events called in one Scheduler function, share the same mutex that is used in checking if method calls don't overlap, but the mutex generates an unique identifier for each command, including differing parameters, and based on the time.

Your scheduled jobs will expire after 24 hours, which means that with jobs that take 2 hours to complete, you'll get about 12 jobs in a day completed. More if the jobs are small, less if the jobs take longer. This is because PHP is a single threaded process by default. First task 1, then task 2, then task 3. Etc... This means that if each tasks takes 2 hours, then after 12 tasks their queued jobs expire because the job has been running for 1440 minutes and then the new jobs are scheduled and it starts again from the top.

Luckily there is a way to make sure they run simultaneously .

I suggest you add ->runInBackground() to your scheduling calls.

$schedule->command('rollOverCron:send ' . $country)
        ->everyTenMinutes()
        ->withoutOverlapping()
        ->runInBackground()
        ->emailOutputTo(['ext.amourya@cma-cgm.com','EXT.KKURANKAR@cma-cgm.com']);cgm.com']);
    }