504 Gateway timeout while generating excel file

Question

I'm trying to implement excel export for some amount of data. After 5 minutes I receive a 504 Gateway timeout. In the backend the process continues with its work.

For the whole service to finish, I need approximately 15 minutes. Is there anything I can do to prevent this? I dont have access to the servers in production.

The app is Spring boot with Oracle database. I'm using POI for this export.

Answer 1

One common way to handle these kinds of problems is to have the first request start the process in the background, and when the file has been generated, download the results from another place. The first request finishes immediately, and the user can then check another view to see if the file has been generated, and download the results.

Answer 2

You can export the data in smaller chunks. Run a test with say 10K records, make a note of the id of the last record and repeat the export starting at the next record. If 10K finishes quickly, then try 50K. If you have a timer that might come in handy. Good luck.

Answer 3

I had the same situation where the timeout of the network calls wasn't in our hand, so I guess you have something where it is 5 mins to receive the 1st byte and then the timeout is gone.

My solution was, let's assume you have a controller and a query layer to talk to the database. In this case, you make your process in the Async way. The call to this controller should just trigger that async execution and return the success status immediately, without waiting. Here execution will happen in the background. Futures can be used here as they are async and you can also handle the result once completed by using callback methods of Future.

You can implement using Future and callback methods in java8 like below:

Futures.addCallback(
  exportData,
  new FutureCallback<String>() {
    public void onSuccess(String message) {
      System.out.println(message);
    }
    public void onFailure(Throwable thrown) {
      thrown.getCause();
    }
  }, 
service)

and in Scala like:

 val result = Future {
    exportData(data)
 }

 result.onComplete {
    case Success(message) => println(s"Got the callback result:  
    $message")
    case Failure(e) => e.printStackTrace
 }