Connection time out because of delayed response by node.js

Question

I have an ejs page which initiates a GET request to execute a shell script.

And node js responds to the page after executing the script with the output of shell script

res.render{
    output: shell_script_output
}

Script gets executed correctly and prints the output in node.js console.

But I get a connection time out error after 2 mins in my browser because the script executes for more than 5 mins.

What may be optimal solution for this? How to make the ejs page to wait till the script gets executed?

Thanks.

Answer 1

Instead of running the script, put the details of the request on a job queue (which could be a database table or, if you want to get complicated, something like ZeroMQ) and return immediately (passing the id of the job in the response).

Have a separate process watch the queue and trigger the shell script when there is a job on it. Once the script finishes, put the results into a response queue.

Meanwhile, back on the client, poll the server periodically (say every 30 seconds) with a request for the job id. Have the server check the response queue to see if that data for the job exists then either respond with "Here it is", "It isn't ready yet" or "It's been 15 minutes, something has probably gone wrong".