FetchError while deleting files Google Cloud Storage via Nodejs Firebase functions

Question

I am having a fetchError while trying to delete multiple files in a single bucket. My application requires finding older objects and then deleting any files a user has uploaded to the cloud storage.

So when I loop through files to delete, I seem to get a fetchError when there are too many files to delete.

At first it was working fine(testing phase) but now I notice that I get a fetchError when there is too many files to delete at the same time.

error:

{ FetchError: request to https://storage.googleapis.com/storage/v1/b/bucketName/o/filePath? failed, reason: socket hang up
    at ClientRequest.<anonymous> (/srv/node_modules/node-fetch/lib/index.js:1453:11)
    at emitOne (events.js:116:13)
    at ClientRequest.emit (events.js:211:7)
    at TLSSocket.socketErrorListener (_http_client.js:401:9)
    at emitOne (events.js:116:13)
    at TLSSocket.emit (events.js:211:7)
    at emitErrorNT (internal/streams/destroy.js:66:8)
    at _combinedTickCallback (internal/process/next_tick.js:139:11)
    at process._tickDomainCallback (internal/process/next_tick.js:219:9)
  message: 'request to https://storage.googleapis.com/storage/v1/b/bucketName/o/filePath? failed, reason: socket hang up',
  type: 'system',
  errno: 'ECONNRESET',
  code: 'ECONNRESET' }

my deleteFunction:

I query a collection, then loop the querysnapshot to get the individual documentsnapshots where I get the image and video filePaths which I pass into deleteFile(myBucketName, mediaFilePath)

const gcs = new Storage();

async function deleteFile(bucketName, fileName) {

 await gcs.bucket(bucketName).file(fileName).delete();

}

I am thinking, should I update the meta-data of the images to be deleted and listen in another function for onMetaDataChange for storage and then delete it? Would that be any better?

Answer 1

This doesn't seem to be an issue with Storage or Cloud Functions. Seems to be an issue with your implementation to handle multiple-requests as the error is related to the socket. ( Here's is explained better). You can use batch requests or this answer could help you.