简体   繁体   English

节点 JS:如何在 function 的 promise 中正确结束/销毁 stream

[英]Node JS: How to properly end/destroy a stream within a promise of a function

GOAL目标

I would like to know why on.destroy() will no longer free my memory if a promise is within a function.我想知道如果 promise 在 function 内,为什么 on.destroy() 将不再释放我的 memory。

Secondly, I would like to know either a proper way to on.destroy a promise within a function or pass values to promise without requiring a function. Secondly, I would like to know either a proper way to on.destroy a promise within a function or pass values to promise without requiring a function.

It's easy to get a promise to end/destroy if it is not in a function - But I need to pass info to the promise object and don't know any other way of doing that without wrapping a function around it. It's easy to get a promise to end/destroy if it is not in a function - But I need to pass info to the promise object and don't know any other way of doing that without wrapping a function around it. The problem is once the function is wrapped around the promise , the end/destroy call of the promise is no longer detected.问题是一旦 function 包裹在 promise 周围,就不再检测到 promise 的结束/销毁调用。

THIS WORKS : I can correctly end a stream within a promise with the code below:有效:我可以使用以下代码在 promise 中正确结束 stream:

const p1= new Promise((resolve, reject) => {
 
  let readStream = readline.createInterface({
    input: fs.createReadStream('pathtofile.txt','utf8')
  });

  readStream.on("line", (line) => {
    //READ LARGE FILE HERE, LINE BY LINE
  });
    
  readStream.on('end', () => {
    readStream.destroy(); /*frees memory*/
  });

  readStream.on("close", () =>
    resolve({
      RETURNVALUE
    }) 
  )
});

Promise.all([p1]).then((results) => {console.log(results)};

THIS DOESN'T WORK: If I wrap a function around promise to pass values, .on end/destroy no longer works (thus heap errors are thrown):这不起作用:如果我将 function 包裹在 promise 周围以传递值,.on end/destroy 不再起作用(因此引发堆错误):

const p1 = function(value1,value2,value3){
   return new Promise((resolve, reject) => {
     let readStream = readline.createInterface({
       input: fs.createReadStream('pathtofile.txt','utf8')
     });
    
    readStream.on("line", (line) => {
      //READ LARGE FILE HERE, LINE BY LINE
    });
        
    readStream.on('end', () => {
      readStream.destroy();   /*No longer frees memory*/
    });
    
    readStream.on("close", () =>
      resolve({
        RETURNVALUE
      }) 
    )
  });
}
    
Promise.all([p1(v1,v2,v3]).then((results) => {console.log(results)};

Turns out the coding above was entirely correct and I was wrong as to the reason why my script was ending up with EMFILE errors such as "too many files are open".事实证明,上面的编码是完全正确的,而我的脚本最终出现 EMFILE 错误的原因是错误的,例如“打开的文件太多”。 The issue was that I am on MAC OS Monterey with a default of 256 limit of file descriptors that can be opened at one time.问题是我在 MAC OS Monterey 上,默认可以一次打开的文件描述符限制为 256 个。 I upped my ulimit -n from 256 to 150000 and everything worked fine.我将 ulimit -n 从 256 提高到 150000,一切正常。

I used the resource below to up the limit:

https://superuser.com/questions/1634286/how-do-i-increase-the-max-open-files-in-macos-big-sur https://superuser.com/questions/1634286/how-do-i-increase-the-max-open-files-in-macos-big-sur

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM