how to download large files chunk by chunk using large files using angular.js and java

Question

We have web application with client in agnular.js and server in java spring. I am working on functionality of downloading this log file ie logs.tar from client.

Currently we are using blob to download. Our issue is in case this log size becomes huge like greater than 2GB then while streaming it will create load on application memory. so i want way to download large files chunk by chunk and not required to load entire blob into memory. please suggest way out.

Server side java code -

   public ResponseEntity<?> downloadLogs(HttpServletRequest request) {
        File file = preferencesService.downloadLogs();
        if (file != null) {
            FileInputStream inputStream;
            try {
                inputStream = new FileInputStream(file);
                byte[] content = FileCopyUtils.copyToByteArray(inputStream);

                String filename = "com-logs.tar";
                HttpHeaders responseHeaders = new HttpHeaders();
                responseHeaders.add(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=" + filename);
                responseHeaders.add(HttpHeaders.CONTENT_TYPE, "application/octet-stream"

);

        return new ResponseEntity<byte[]>(content, responseHeaders, HttpStatus.OK);

        } catch (Exception e) {
            logger.error("Error while processing log file for download", e);
        }
    } else {
        logger.error("Failed to download logs");
    }
    return ResponseEntity.badRequest().build();
} 

Client side Angular.js code -

this._service.downloadLogs().subscribe(
            success => {               
                var blb = new Blob([success], { 'type': "application/octet-stream" });
                if (window.navigator && window.navigator.msSaveOrOpenBlob) {
                    window.navigator.msSaveOrOpenBlob(blb, 'logs.tar');
                }
                else {
                    var link = document.createElement('a');
                    link.href = window.URL.createObjectURL(blb);
                    link.download = "logs.tar";
                    document.body.appendChild(link);
                    link.click();
                    document.body.removeChild(link);
                }
            });

New Server side java code -

public void downloadLogs(HttpServletResponse resonse) {
        File file = preferencesService.downloadLogs(id);       
        if (file != null) {
            try {
                resonse.setContentType("application/octet-stream");
                resonse.setHeader("Content-Disposition", "attachment;filename=" + file.getName());
                BufferedInputStream inStrem = new BufferedInputStream(new FileInputStream(file));
                BufferedOutputStream outStream = new BufferedOutputStream(resonse.getOutputStream());

                byte[] buffer = new byte[1024];
                int bytesRead = 0;
                while ((bytesRead = inStrem.read(buffer)) != -1) {
                    outStream.write(buffer, 0, bytesRead);
                }
                outStream.flush();
                inStrem.close();

            } 
            ...
        }

Answer 1

The important thing is to not read the file into memory, but to pass the stream on:

public ResponseEntity<?> downloadLogs(HttpServletRequest request) {
    File file = preferencesService.downloadLogs();
    if (file != null) {
        try (InputStream inputStream = Files.newInputStream(file.toPath())) {
            InputStreamResource inputStreamResource =
                    new InputStreamResource(new inputStream);
            HttpHeaders responseHeaders = new HttpHeaders();
            //responseHeaders.setContentLength(Files.size(file.toPath()));
            responseHeaders.add(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename="
                    + filename);
            responseHeaders.add(HttpHeaders.CONTENT_TYPE, "application/octet-stream");
            return new ResponseEntity(inputStreamResource, responseHeaders, HttpStatus.OK);
        }
    }
    ...
}

Consider compression as this will hugely speed things up and cause less server load. Chunking, setting content length, deflate compression web filters, and so on should be looked into.