如何使用 Node.js 从 Dropbox 下载大文件？

Question

I want to implement a big file downloading (approx. 10-1024 Mb). I've already succeeded to get a file from Dropbox:

operationResult = await dbx.filesDownload({
    path: `/${CONFIG_STORAGE.uploader.assetsPath}/${fileUUID}`
});

Then I bundle the received file with a meta-data and I return it to my Node.js server:

fileMIME = mime.lookup(operationResult.name);

const downloadResult = Object.freeze({
    fileBinary: operationResult.fileBinary,
    fileLength: operationResult.fileBinary.length,
    fileMIME,
    fileName: operationResult.name,
    isSucceeded,
    message
});

return downloadResult;

Now I convert a Buffer , I got from Dropbox, into a Readable stream and pipe it back to a client:

res.setHeader("Content-Disposition", "attachment; filename=" + downloadResult.fileName);
res.setHeader("Content-Type", downloadResult.fileMIME);

const fileReadableStream = new Readable();

fileReadableStream.push(downloadResult.fileBinary);
fileReadableStream.push(null);

fileReadableStream.pipe(res);

Up until now everything is clear and works. Here I face a first pitfall: I need somehow to trigger a download process in browser.

In many examples, some small image or JSON are used, which we can completely load into RAM, make operations, eg transforming to Base64 , assign it to a.href , and trigger a.click() . But since my file is 10-50 Mb I'm not sure if such approach is a correct one.

I've already tried Fetch API:

const response = await fetch(`${host}/download?fileName=${fileName}`, {
    credentials: "same-origin",
    method: "POST",
    mode: "cors"
});

const a = document.createElement("a");
a.href = response.text();
a.download = "MyFile.pdf";
a.click();

But I always get Failed - No file error. I also tried to use jQuery AJAX, and XMLHttpRequest ( XHR ), but still no file is downloaded.

Perhaps, there is something I'm missing. How to get a 10-1024 Mb file from a server?

PS I never thought that such a trivial task, as a file downloading, can be so complicated.

Answer 1

I've solved the issue by switching from filesDownload to filesGetTemporaryLink , which returns a link to a file instead of the file itself. Then I trigger a downloading of this link.

The final result:

operationResult = await dbx.filesGetTemporaryLink({
    path: `/${CONFIG_STORAGE.uploader.assetsPath}/${fileUUID}`
});

const downloadResult = Object.freeze({
    fileLength: operationResult?.metadata.size,
    fileLink: operationResult?.link,
    fileMIME: mime.lookup(operationResult?.metadata.name),
    fileName: operationResult?.metadata.name,
    isSucceeded,
    message
});

return downloadResult;

Then I send the output to a client:

res.json(downloadResult);

On a client-side I get it via await / async Fetch API call:

const fileResponse = await fetch(``${host}/downloadDocument`, {
    body: JSON.stringify({fileUUID: fileName}),
    cache: "no-cache",
    credentials: "same-origin",
    headers: {
        "Content-Type": "application/json"
    },
    method: "POST",
    mode: "cors"
});

const fileData = await fileResponse.json();

const aTag = document.createElement("a");

aTag.href = fileData.fileLink;
aTag.download = fileData.fileName;
aTag.click();

As a result, a server doesn't have to deal with files at all, no extra CPU, RAM, or traffic impact, no matter what size of a file I'm trying to download.