PHP Long running request executed twice

Question

I am struggling with a problem occuring only on very long running requests. I have a web application with the possibility to export a large amount of data on demand in a XML file. Depending on the data the export runs from just a second to several minutes. I tested the export with little data and the xml generation and file download worked well. Then I tested it with a large amount of data and the xml generation ran about 30 minutes (due to lots of database queries). But the problem for this long running request is that after generating the xml file it is not downloading but generating the file again. While doing the xml generation again the gets an site loading error. The same happened with more data and a xml generation time of more than one hour... So why is this happening? This problem occurs only for such requests running about 30 minutes or longer. Is it possible that there is some timeout which starts the request again? But the second file generation starts exactly after finishing the first and not after a fixed duration.

Answer 1

But the second file generation starts exactly after finishing the first and not after a fixed duration.

If you use file based sessions, the lock on the session file will only allow one running PHP script to access that file at one time, which could account for the blocking and why the next request happens as soon as the current script finishes.

For big exports, one option is to process "offline" in the backend somehow and check for the finished file. Eg: Request a download, get a "key" back right away, and have php spawn the export in the backend. The browser can then keep checking if the export for "key" has finished, and download it when it's ready. Allowing the user to start the export and not worry about the browser closing mid way or download the same export multiple times without regenerating.

Alternatively, if the export is needed regularly, just preprocess it from a cron job so the end user can just download the fresh data quickly when they want it without waiting.