简体   繁体   中英

Downloading multiple files from Google cloud storage using nodejs fails without errors

My code successfully downloads some of the files but not all. Execution just stops in the middle without any errors and the last file may be only partially downloaded. It always fails to download some files even if I change the number of files or if I use different files. File size doesn't seem to matter.

I've tried many things but it seems that I'm not able to catch any exceptions or errors when it stops. I've tried using try-catch and process.on events but not able to catch anything.

I'm pretty sure few months ago I used this kind of code to download hundreds of files without any problems.

Here is a simplified version of my current code.

const { Storage } = require('@google-cloud/storage');
const storage = new Storage({ keyFilename: 'D:/myProject/myKeyFile.json' });

var folder = 'D:/myProject/downloadedFiles';
var bucketName = 'bucket_1';

async function downloadFile(fileName) {
  var fullPath = folder + '/' + fileName;
  const options = {
    destination: fullPath,
  };
  await storage.bucket(bucketName).file(fileName).download(options);
  console.log(`gs://${bucketName}/${fileName} downloaded to ${fullPath}.`);
}

async function downloadFiles() {
  var filenames = ['file1', 'file2', 'file3', 'file4', 'file5', 'file6'];

  for(var i = 0; i < filenames.length; i++){
    await downloadFile(filenames[i], i).catch(console.error);
  }
}
downloadFiles().catch(console.error);

It turns out the download only fails on that specific computer, so my code is not the problem. My guess it is.network related, maybe because a.network switch was replaced some time ago.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM