[英]NodeJS: Reached heap limit Allocation failed - JavaScript heap out of memory
I have a VPS (Debian) with 2 GB RAM where I run pm2
in order to execute a node.js server application.我有一个带有 2 GB RAM 的 VPS (Debian),我在其中运行pm2
以执行 node.js 服务器应用程序。 I have a cache and every night, at 3 am I want to dump it into a file as a backup.我有一个缓存,每天晚上凌晨 3 点,我想将它转储到一个文件中作为备份。
Every day at 3, I got the following error:每天凌晨 3 点,我都会收到以下错误:
<--- Last few GCs ---> <--- 最后几次 GC --->
[717119:0x63345b0] 66921944 ms: Mark-sweep 956.0 (1043.7) -> 954.9 (1043.7) MB, 737.3 / 0.0 ms (average mu = 1.000, current mu = 0.349) allocation failure; [717119:0x63345b0] 66921944 ms:Mark-sweep 956.0 (1043.7) -> 954.9 (1043.7) MB,737.3 / 0.0 ms(平均 mu = 1.000,当前 mu = 0.349)分配失败; scavenge might not succeed [717119:0x63345b0] 66922975 ms: Mark-sweep 962.8 (1043.7) -> 961.9 (1059.7) MB, 978.0 / 0.0 ms (average mu = 0.999, current mu = 0.052) allocation failure;清除可能不会成功 [717119:0x63345b0] 66922975 毫秒:标记扫描 962.8 (1043.7) -> 961.9 (1059.7) MB,978.0 / 0.0 毫秒(平均 mu = 0.999,当前 mu = 0.052)分配失败; scavenge might not succeed清除可能不会成功
<--- JS stacktrace ---> <--- JS 堆栈跟踪 --->
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory 1: 0xb6e500 node::Abort() [node /home/userName/web/appName.com/public_html/appNameServer/appNameServer.js] 2: 0xa7e632 [node /home/userName/web/appName.com/public_html/appNameServer/appNameServer.js] 3: 0xd47f20 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node /home/userName/web/appName.com/public_html/appNameServer/appNameServer.js] 4: 0xd482c7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node /home/userName/web/appName.com/public_html/appNameServer/appNameServer.js] 5: 0xf25685 [node /home/userName/web/appName.com/public_html/appNameServer/appNameServer.js] 6: 0xf37b6d v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node /home/userName/web/appName.com/public_html/appNameServer/stoc> 7: 0xf1226e v8::internal::HeapAllocator::AllocateRawWithLightRet致命错误:达到堆限制分配失败 - JavaScript 堆超出 memory 1:0xb6e500 node::Abort() [node /home/userName/web/appName.com/public_html/appNameServer/appNameServer.js] 2:0xa7e632 [node / home/userName/web/appName.com/public_html/appNameServer/appNameServer.js] 3: 0xd47f20 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [节点/home/userName /web/appName.com/public_html/appNameServer/appNameServer.js] 4: 0xd482c7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [节点/home/userName /web/appName.com/public_html/appNameServer/appNameServer.js] 5: 0xf25685 [节点/home/userName/web/appName.com/public_html/appNameServer/appNameServer.js] 6: 0xf37b6d v8::internal::Heap: :CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [节点/home/userName/web/appName.com/public_html/appNameServer/stoc> 7: 0xf1226e v8::internal ::HeapAllocator::AllocateRawWithLightRet rySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node /home/userName/web/app> 8: 0xf13637 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node /home/userName/web/stoc> 9: 0xef3b80 v8::internal::Factory::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [node /home/userName/web/appName.com/public_html/appNameServer/appNameServer.js] 10: 0xeeb5f4 v8::internal::FactoryBase<v8::internal::Factory>::AllocateRawWithImmortalMap(int, v8::internal::AllocationType, v8::internal::Map, v8::internal::AllocationAlignment) [node /home/userName/web/stoc>11: 0xeed988 v8::internal::FactoryBase<v8::internal::Factory>::NewRawTwoByteString(int, v8::internal::AllocationType) [node /home/userName/web/appName.com/public_html/appNameServer/appNameServer.js] 12: 0x131ddec v8::internal::Inc rySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [节点/home/userName/web/app> 8: 0xf13637 v8::internal::HeapAllocator: :AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [节点/home/userName/web/stoc> 9: 0xef3b80 v8::internal::Factory ::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [node /home/userName/web/appName.com/public_html/appNameServer/appNameServer.js] 10: 0xeeb5f4 v8::internal ::FactoryBase<v8::internal::Factory>::AllocateRawWithImmortalMap(int, v8::internal::AllocationType, v8::internal::Map, v8::internal::AllocationAlignment) [节点/home/userName/web /stoc>11: 0xeed988 v8::internal::FactoryBase<v8::internal::Factory>::NewRawTwoByteString(int, v8::internal::AllocationType) [节点/home/userName/web/appName.com/public_html /appNameServer/appNameServer.js] 12: 0x131ddec v8::internal::Inc rementalStringBuilder::Extend() [node /home/userName/web/appName.com/public_html/appNameServer/appNameServer.js] 13: 0x10349d8 v8::internal::JsonStringifier::SerializeString(v8::internal::Handle<v8::internal::String>) [node /home/userName/web/appName.com/public_html/appNameServer/appNameServer.js] 14: 0x1035c52 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [node /home/di>15: 0x10378af v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [node /home/di>16: 0x10378af v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [node /home/di>17: 0x10378af v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize rementalStringBuilder::Extend() [节点/home/userName/web/appName.com/public_html/appNameServer/appNameServer.js] 13: 0x10349d8 v8::internal::JsonStringifier::SerializeString(v8::internal::Handle<v8 ::internal::String>) [node /home/userName/web/appName.com/public_html/appNameServer/appNameServer.js] 14: 0x1035c52 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier: :Serialize_(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [节点/home/di>15: 0x10378af v8 ::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8:: internal::Object>) [node /home/di>16: 0x10378af v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_(v8::internal::Handle<v8::internal: :Object>, bool, v8::internal::Handle<v8::internal::Object>) [节点/home/di>17: 0x10378af v8::internal::JsonStringifier::Result v8::internal::JsonStringifier ::连载_(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [node /home/di>18: 0x10378af v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [node /home/di>19: 0x10378af v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [node /home/di>20: 0x10378af v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [node /home/di>21: 0x1037f13 v8::internal::JsonStringifier::SerializeJSReceiverSlow(v8::internal::Handle<v8::internal::JSReceiver>) [node /home/userName/web/appName.com/public_html/appNameServer/appNameServer.js] 22: 0x1039897 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier: _(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [node /home/di>18: 0x10378af v8: :internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal ::对象>) [节点/home/di>19: 0x10378af v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_(v8::internal::Handle<v8::internal::对象>, bool, v8::internal::Handle<v8::internal::Object>) [node /home/di>20: 0x10378af v8::internal::JsonStringifier::Result v8::internal::JsonStringifier: :Serialize_(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [节点/home/di>21: 0x1037f13 v8 ::internal::JsonStringifier::SerializeJSReceiverSlow(v8::internal::Handle<v8::internal::JSReceiver>) [节点/home/userName/web/appName.com/public_html/appNameServer/appNameServer.js] 22: 0x1039897 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier: :Serialize_(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [node /home/d>23: 0x103ae5f v8::internal::JsonStringify(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>) [node /hom>24: 0xdcdbf7 v8::internal::Builtin_JsonStringify(int, unsigned long*, v8::internal::Isolate*) [node /home/userName/web/appName.com/public_html/appNameServer/appNameServer.js] 25: 0x16e9af9 [node /home/userName/web/appName.com/public_html/appNameServer/appNameServer.js] :Serialize_(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [节点/home/d>23: 0x103ae5f v8 ::internal::JsonStringify(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>) [node /hom>24: 0xdcdbf7 v8::internal::Builtin_JsonStringify(int, unsigned long*, v8::internal::Isolate*) [node /home/userName/web/appName.com/public_html/appNameServer/appNameServer.js] 25: 0x16e9af9 [节点/home/userName/web/appName.com/public_html/appNameServer/appNameServer.js]
I know the error is straight forward and the trivial answer would be to add more RAM, but I still want to share some more data:我知道错误很简单,简单的答案是添加更多 RAM,但我仍然想分享更多数据:
node -e 'console.log(v8.getHeapStatistics().heap_size_limit/(1024*1024))'
the result is 1662.当我运行这个node -e 'console.log(v8.getHeapStatistics().heap_size_limit/(1024*1024))'
结果是 1662。2022-12-27T03:00:00.016Z: exporting to json 2|appName| 2022-12-27T03:00:00.016Z:导出到 json 2|appName|
2022-12-27T03:00:14.747Z: server is running on https at port... 2022-12-27T03:00:14.747Z:服务器正在端口 https 上运行...
Based on the numbers above, 640mb (gross) and the fact that there's at least one more 1gb left for node, I don't understand why there's a heap out of memory in the first place.根据上面的数字,640mb(毛)和节点至少还有 1gb 的事实,我不明白为什么一开始会有 memory 堆。
Please advise, thanks请指教,谢谢
In my case就我而言
$ node -e 'console.log(v8.getHeapStatistics().heap_size_limit/(1024*1024))'
4144
but但
$ export NODE_OPTIONS=--max_old_space_size=8192 && node -e 'console.log(v8.getHeapStatistics().heap_size_limit/(1024*1024))'
8240
so maybe you can modify amount of available memory using this variable.所以也许您可以使用此变量修改可用 memory 的数量。
Probably you are writing file using可能您正在使用
fs.writeFileSync('/tmp/test.txt', content);
but maybe you can switch to但也许你可以切换到
fs.createWriteStream(path, [options])
then you will not have to store all content of file in memory in the same time.那么您就不必同时将文件的所有内容存储在 memory 中。
I solved similar cases using tools like this:我使用这样的工具解决了类似的情况:
https://www.jetbrains.com/help/webstorm/v8-cpu-and-memory-profiling.html#ws_node_cpu_profiling https://www.jetbrains.com/help/webstorm/v8-cpu-and-memory-profiling.html#ws_node_cpu_profiling
in console you can simply start your script with --porf
flag.在控制台中,您可以简单地使用--porf
标志启动脚本。 It will created huge, hard to read, but very detailed logs telling about memory allocation.它将创建巨大的、难以阅读但非常详细的日志,讲述 memory 分配。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.