allocation failed - javascript heap out of memory in loopback.js

Question

I'm tryping to fetch large data through loopback.js and i'm getting error FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

<--- Last few GCs --->



45903 ms: Mark-sweep 1385.6 (1438.9) -> 1385.6 (1438.9) MB, 742.8 / 0.0 ms (+ 0.8 ms in 1 steps since start of marking, biggest step 0.8 ms) [allocation failure] [GC in old space requested].
   46512 ms: Mark-sweep 1385.6 (1438.9) -> 1387.3 (1422.9) MB, 607.4 / 0.0 ms (+ 0.8 ms in 1 steps since start of marking, biggest step 0.8 ms) [last resort gc].
   47111 ms: Mark-sweep 1387.3 (1422.9) -> 1389.5 (1422.9) MB, 599.6 / 0.0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x1c6b6bcfb51 <JS Object>
    1: deserializeObject(aka deserializeObject) [/home/ubuntu/veerendergoudapi/node_modules/bson/lib/bson/parser/deserializer.js:~41] [pc=0x1a9fab83915a] (this=0x1c6b6b04381 <undefined>,buffer=0x193d54f18579 <an Uint8Array with map 0xd4eba2066c9>,index=0,options=0x1e11b6cedfc9 <an Object with map 0x19eb230b3809>,isArray=0x1c6b6b04381 <undefined>)
    2: deserialize(aka deserialize) [/home/ubuntu...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]
 2: 0x10a3b3c [node]
 3: v8::Utils::ReportApiFailure(char const*, char const*) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewTransitionArray(int) [node]
 6: v8::internal::TransitionArray::Insert(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Map>, v8::internal::SimpleTransitionFlag) [node]
 7: v8::internal::Map::CopyReplaceDescriptors(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::DescriptorArray>, v8::internal::Handle<v8::internal::LayoutDescriptor>, v8::internal::TransitionFlag, v8::internal::MaybeHandle<v8::internal::Name>, char const*, v8::internal::SimpleTransitionFlag) [node]
 8: v8::internal::Map::CopyAddDescriptor(v8::internal::Handle<v8::internal::Map>, v8::internal::Descriptor*, v8::internal::TransitionFlag) [node]
 9: v8::internal::Map::CopyWithField(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::FieldType>, v8::internal::PropertyAttributes, v8::internal::Representation, v8::internal::TransitionFlag) [node]
10: v8::internal::Map::TransitionToDataProperty(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::StoreFromKeyed) [node]
11: v8::internal::LookupIterator::PrepareTransitionToDataProperty(v8::internal::Handle<v8::internal::JSObject>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::StoreFromKeyed) [node]
12: v8::internal::Object::AddDataProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::ShouldThrow, v8::internal::Object::StoreFromKeyed) [node]
13: v8::internal::Object::SetProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::LanguageMode, v8::internal::Object::StoreFromKeyed) [node]
14: v8::internal::Runtime::SetObjectProperty(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::LanguageMode) [node]
15: v8::internal::Runtime_SetProperty(int, v8::internal::Object**, v8::internal::Isolate*) [node]
16: 0x1a9faac092a7
error: Forever detected script was killed by signal: SIGABRT
error: Script restart attempt #571

Answer 1

We faced a similar issue couple of days back when our application was in production and there were around 200 active users. The entire application went down because of this. After hours of debugging, we figured out that one of the APIs were not passing the right error object to the loopback remote method callback.

Loopback expects the error object to be of the [format][1]

{status:<some statuscode>, message:<error message>, name:<name for the error>}

and we were passing it the actual exception and everything went haywire. After pushing this fix, the leak and error have gone away.

Please do verify that you have handled all the error objects properly.

If you have handled the errors properly then fetching the large data set may be overflowing the heap. Please try increasing that.

node --max-old-space-size=4096 file.js