从数组中的链接下载文件

Question

I'm trying to download files from links in an array with a length of several thousand positions. The problem is that when I iterate over the array I hit a wall when trying to synchronize the file fetch and write ( Maximum call stack size exceeded). I've tried to make a recursive function and played with promises but I still haven't managed to find a solution. Help please!

My code so far:

 function download(url, dest, cb) { return new Promise(function (resolve, reject) { let request = https.get(url, function (response) { let file = fs.createWriteStream(dest); response.pipe(file); file.on('finish', function () { console.log('File downloaded') resolve(file.close(cb)); }); }).on('error', function (err) { reject(err) }); }) }; function recursiveDownload(links, i) { if (i < links.length) { download(links[i], './data/' + i + '.csv') .then(recursiveDownload(links, ++i)) } else { console.log('ended recursion') } } recursiveDownload(links, 0) 

Answer 1

You can use a for loop to serialize all your promises and synchronize them. Try the following:

var promise = Promise.resolve();
for(let i = 0; i < links.length; i++){
  promise = promise.then(()=> download(links[i], './data/' + i + '.csv'));
}

Or you can even chain your promises using Array.reduce() :

var promise = links.reduce((p, link, index) => p.then(()=>download(link, './data/' + index + '.csv')),Promise.resolve());

Answer 2

Because you call recursiveDownload immeadiately withou waiting for the download to succeed. You actually want to call it when .then calls back:

 download(links[i], './data/' + i + '.csv')
        .then(() => recursiveDownload(links, ++i))