[英]FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory when processing large files with fs
I have a nodeJs script that process a bunch of large .csv files (1.3GB for all).我有一个 nodeJs 脚本,它处理一堆大型 .csv 文件(全部为 1.3GB)。 It run for a moment and throw this error:
它运行了一会儿并抛出此错误:
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
I have tried to put #!/usr/bin/env node --max-old-space-size=4096
at the beginning of my file but this didn't solve my problem...我试图将
#!/usr/bin/env node --max-old-space-size=4096
放在文件的开头,但这并没有解决我的问题......
Tried to google it but nearly no pertinent info about my issue..试图谷歌它,但几乎没有关于我的问题的相关信息..
Can I flush memory once I do not need file content ?一旦我不需要文件内容,我可以刷新内存吗? Do I need to allocate more memory to nodeJs ?
我需要为 nodeJs 分配更多内存吗?
Thanks ;)谢谢 ;)
Here is my code sample:这是我的代码示例:
fs.readdir(dirName, function(err, filenames) {
if (err) console.error(err);
else {
filenames.forEach(function(filename) {
fs.readFile(dirName + filename, 'utf-8', function(err, content) {
if (err) console.error(err);
else processFile(content);
});
});
}
});
I have finally found the solution to the problem !我终于找到了问题的解决方案! I needed to launch the process adding the
--max-old-space-size=8192
param to node process like so:我需要启动将
--max-old-space-size=8192
参数添加到节点进程的进程,如下所示:
node --max-old-space-size=8192 ./myScript.js
I processed my 1.3GB without a problem !!我处理了我的 1.3GB 没有问题!!
You can increase the amount of memory allocated to the command by running the following command prior to running Snyk:您可以通过在运行 Snyk 之前运行以下命令来增加分配给该命令的内存量:
Linux/macOS Linux/macOS
export NODE_OPTIONS=--max-old-space-size=8192 For example: export NODE_OPTIONS=--max-old-space-size=8192 例如:
export NODE_OPTIONS=--max-old-space-size=8192 snyk test export NODE_OPTIONS=--max-old-space-size=8192 snyk monitor export NODE_OPTIONS=--max-old-space-size=8192 snyk 测试 export NODE_OPTIONS=--max-old-space-size=8192 snyk 监视器
Windows视窗
From the control panel go to System -> Advanced system settings -> Environment Variables -> New (user or system)从控制面板转到系统 -> 高级系统设置 -> 环境变量 -> 新建(用户或系统)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.