NodeJs process running out of memory on aws

Question

I am running a nodeJS app on aws t2micro instance with Mongodb. This is how my index.js file on server looks like

const db = require('./src/db');
const coins = require('./src/scrapers/populate-coins'); 
const scrape = require('./src/scrapers/index.js');
const settings = require('./src/trading/init-settings.js');
const trading = require('./src/trading/index.js');
const server = require('./src/index.js');

Here scrape and trading are processes that run within a setInterval while server handles the API request. I am using a angular frontend and have deployed the whole thing on single aws t2micro instance with 1GB RAM and 16GB storage. Things went fine for couple of days until data in db was insignificant. Once I had >10K rows of data in one of my tables, API calls started taking a lot of time, as much as 1 minute. In the dev tools it showed the waiting(TTFB) took ~40s. Upon checking the logs I am getting this

0|trading  | Warning: connect.session() MemoryStore is not
0|trading  | designed for a production environment, as it will leak
0|trading  | memory, and will not scale past a single process.
0|trading  | FATAL ERROR: Committing semi space failed. Allocation failed - process out of memory
0|trading  |  1: node::Abort() [node /home/ec2-user/app/bc-trading/index.js]
0|trading  |  2: 0x13740dc [node /home/ec2-user/app/bc-trading/index.js]
0|trading  |  3: v8::Utils::ReportOOMFailure(char const*, bool) [node /home/ec2-user/app/bc-trading/index.js]
0|trading  |  4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node /home/ec2-user/app/bc-trading/index.js]
0|trading  |  5: v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [node /home/ec2-user/app/bc-trading/index.js]
0|trading  |  6: v8::internal::Heap::CollectGarbage(v8::internal::GarbageCollector, v8::internal::GarbageCollectionReason, char const*, v8::GCCallbackFlags) [node /home/ec2-user/app/bc-trading/index.js]
0|trading  |  7: v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [node /home/ec2-user/app/bc-trading/index.js]
0|trading  |  8: v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [node /home/ec2-user/app/bc-trading/index.js]

Can anyone help me figure out what is going on here? Help will be highly appreciated.

Answer 1

I'd recommend adding logging to check if the size of the object processed is large or execute the function and monitor the server memory/cpu usage, it could be either mongoDb or the node process that is using all the resources and therefore causing the error/slow responses and also log the server metrics before the start and after the end of each function.
You should also consider that running angular , node and mongoDb on 1 gb of ram will cause issues if there are enough records in the db even if it is a dev environment, the node garbage collector might not be able to free up memory and you will end up with a buildup that will slow everything down.