简体   繁体   中英

Node JS: How to properly end/destroy a stream within a promise of a function

GOAL

I would like to know why on.destroy() will no longer free my memory if a promise is within a function.

Secondly, I would like to know either a proper way to on.destroy a promise within a function or pass values to promise without requiring a function.

It's easy to get a promise to end/destroy if it is not in a function - But I need to pass info to the promise object and don't know any other way of doing that without wrapping a function around it. The problem is once the function is wrapped around the promise , the end/destroy call of the promise is no longer detected.

THIS WORKS : I can correctly end a stream within a promise with the code below:

const p1= new Promise((resolve, reject) => {
 
  let readStream = readline.createInterface({
    input: fs.createReadStream('pathtofile.txt','utf8')
  });

  readStream.on("line", (line) => {
    //READ LARGE FILE HERE, LINE BY LINE
  });
    
  readStream.on('end', () => {
    readStream.destroy(); /*frees memory*/
  });

  readStream.on("close", () =>
    resolve({
      RETURNVALUE
    }) 
  )
});

Promise.all([p1]).then((results) => {console.log(results)};

THIS DOESN'T WORK: If I wrap a function around promise to pass values, .on end/destroy no longer works (thus heap errors are thrown):

const p1 = function(value1,value2,value3){
   return new Promise((resolve, reject) => {
     let readStream = readline.createInterface({
       input: fs.createReadStream('pathtofile.txt','utf8')
     });
    
    readStream.on("line", (line) => {
      //READ LARGE FILE HERE, LINE BY LINE
    });
        
    readStream.on('end', () => {
      readStream.destroy();   /*No longer frees memory*/
    });
    
    readStream.on("close", () =>
      resolve({
        RETURNVALUE
      }) 
    )
  });
}
    
Promise.all([p1(v1,v2,v3]).then((results) => {console.log(results)};

Turns out the coding above was entirely correct and I was wrong as to the reason why my script was ending up with EMFILE errors such as "too many files are open". The issue was that I am on MAC OS Monterey with a default of 256 limit of file descriptors that can be opened at one time. I upped my ulimit -n from 256 to 150000 and everything worked fine.

I used the resource below to up the limit:

https://superuser.com/questions/1634286/how-do-i-increase-the-max-open-files-in-macos-big-sur

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM