Getting a 403 Error on manually created Cron job?

Question

My current site is private, so it doesn't get enough consistent traffic to rely on the standard wp-cron functionality in Wordpress. I've disabled the built-in cron and set up a real cron job that sends a get request every 15 minutes to https://mywebsite.com/wp-cron.php?doing_wp_cron

For a while, this worked wonderfully. However, I'm starting to get 403 responses on that cron job now. Sometimes it only happens once and resolves itself. Sometimes it gets a 403 response on every request across the span of several hours.

Any ideas what could be causing this?

Also, on a related note I have two sync functions that process a large amount of data on a nightly basis at midnight and 1:00am. I'm wondering if the cron job is failing because these processes take too long to complete. If so, is there a way that I can schedule these functions to occur regularly, but in a way that doesn't tie up the http request? For instance, calling the function asynchronously so it will carry on regardless of the completion of the function?

Any guidance is greatly appreciated!

Answer 1

AFAIK, you can run Wordpress cron jobs without going through the HTTP stack.

After some Googling, I stumbled upon this :

cd /var/www/example.com/htdocs; php wp-cron.php > /dev/null 2>&1