Node.js long term tasks

Question

I have a node.js server which is communicating from a net socket to python socket. When the user sends an asynchronous ajax request with the data, the node server passes it to the python and gets data back to the server and from there to the client.

The problem occurs when the user sends the ajax request: he has to wait for the response and if the python process takes too much time then the ajax is already timed out.

I tried to create a socket server in node.js and a client that connects to the socket server in python with the data to process. The node server responds to the client with a loading screen. When the data is processed the python socket client connects to the node.js socket server and passes the processed data. However the client can not request the processed data because he doesn't know when it's done.

Answer 1

So you have three systems, and an asynchronous request. I solved a problem like this recently using PHP and the box.com API. PHP doesn't allow keeping a connection open indefinitely so I had a similar problem.

To solve the problem, I would use a recursive request. It's not 'real-time' but that is unlikely to matter.

How this works:

1. The client browser sends the "Get my download thing" request to the Node.js server. The Node.js server returns a unique request id to the client browser.
2. The client browser starts a 10 second poll, using the unique request id to see if anything has changed. Currently, the answer is no.
3. The Node.js server receives this and sends a "Go get his download thing" request to the Python server. (The client browser is still polling every 10 seconds, the answer is still no)
4. The python server actually goes and gets his download thing, sticks it in a place, creates a URL and returns that to the Node.js server. (The client browser is still polling every 10 seconds, the answer is still no)

5. The Node.js server receives a message back from the Python server with the URL to the thing. It stores the URL against the request id it started with. At this point, its state changes to "Yes, I have your download thing, and here it is! - URL).

6. The client browser receives the lovely data packet with its URL, stops polling now, and skips happily away into the sunset. (or similar more appropriate digital response).

Hope this helps to give you a rough idea of how you might solve this problem without depending on push technology. Consider tweaking your poll interval (I suggested 10 seconds to start) depending on how long the download takes. You could even get tricky, wait 30 seconds, and then poll every 2 seconds. Fine tune it to your problem.