Execute server-side execution of PHP script via webpage

Question

First of all sorry to post a question that seems to have been flogged to death on SO before. However, none of the questions I have reviewed helped me to solve my specific problem.

I have built a web application that runs an extensive data processing routine in PHP (ie MySQL queries, calculations, etc.).

Depending on the amount of data fed to the app this processing can take quite a long time so the script needs to run server-side and independently from the web front-end.

There is a problem, however. It seems I cannot control the script execution time limit as long as the script is invoked via cgi.

When I run the script via SSH and the command line it works fine for however long it takes to process the data.

But if I use the exec() command in a php script called via the webserver I always ends up with the error End of script output before headers after approximately 45 seconds.

Rather than having to fiddle with server settings (a nightmare in terms of portability) I would like to find a solution that kicks off the script independently from cgi.

Any suggestions?

Answer 1

Don't execute the long script directly from the website (AKA, directly from Apache) because, as you've mentioned, it will block until it finishes and potentially time out. Instead, use the website to schedule a job (an execution of the long script) to be run immediately.

Here is a basic outline of how you can potentially do this:

1. Create a new, small database to store job requests, including fields job_id , processing_status , run_start_time , and more relevant fields
2. Create some Ajax that hits your server and writes a "job request" to this jobs database, set to execute immediately.
3. Add a crontab script or bot that periodically watches for new jobs. If it finds a job that is yet to be processed but has passed the run_start_time , run it using exec() or some other command executor. This way the command won't timeout because it is not being run by Apache, but by the cron daemon.
4. When the command finishes, update the jobs database saying that processing is finished.
5. From your website, write a frontend that allows the user to see if the requested job is finished yet. Once it finishes, it displays some kind of "Done" indicator or something similar.