NodeJS MongDB query

Question

I'm building a webapp with NodeJS and a MongoDB database. Currently the following is frustrating me - I am lacking in understanding the callback/(a)sync element and still learning this so I assume that it has something to do with this aspect.

I have two main modules:

DB.js export - this function works when called by itself:

DB.GetSuggestions: function(searchTerm) {
    return MongoClient.connect(url).then(function(db) {
        var collection = db.collection("AccentPairs");
        collection.updateMany(
            {Unstressed: searchTerm},
            {$inc: {QueryCount: 1}}
        );
        var result = collection.aggregate([
            { $match: { Unstressed: searchTerm } },
            { $group: { _id: {"WordID": "$WordID", "WordName": "$WordName", "Unstressed": "$Unstressed", "Stressed": "$Stressed", "TranslationEn": "$TranslationEn"}, Forms: { $push: "$Field" }}}
        ]).sort( { Chosen: -1} ).toArray();
        db.close();
        return result;
    });
}

TextHandler.js:

var DB = require("../DB");

function ImportText(Text) {
  var fullText = Text.trim();
  var TextObject = {AuthorName: "", Title: "", Words: []};
  var currentWord = "";
  var BOS = true;
  var EOS = false;
  let reg = /([а-яА-ЯЁё́]+)([^а-яА-ЯЁё́]*)/g;
  var result;
  while (result = reg.exec(fullText)) {
    BOS = EOS;
    EOS = (result[2].indexOf(".") > -1);
    currentWord = result[1];
    WordObject = {WordText: result[1], WordID: "0", Options: [], Tail: result[2], BOS: BOS, EOS: EOS};
    TextObject.Words.push(WordObject);
  }
  //This is what doesn't work - GetSuggestions within the loop doesn't work.
  TextObject.Words.forEach(function(wd){

    console.log(wd.WordText);
    DB.GetSuggestions(wd.WordText).then(function(suggestions){
      wd.Options = suggestions;
    });
  });
}

I am trying to iterate over the TextObject.Words array and search the database for suggestions using the GetSuggestions function. Calling GetSuggestions within the loop doesn't work, outside of the loop it works.

Error message: <--- Last few GCs --->

29335 ms: Mark-sweep 1386.5 (1440.6) -> 1386.5 (1440.6) MB, 1156.9 / 0.7 ms [allocation failure] [GC in old space req uested]. 30456 ms: Mark-sweep 1386.5 (1440.6) -> 1387.5 (1424.6) MB, 1120.6 / 0.7 ms [last resort gc]. 31576 ms: Mark-sweep 1387.5 (1424.6) -> 1388.4 (1424.6) MB, 1119.4 / 0.7 ms [last resort gc].

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 000001186EBCFB49 1: /* anonymous /(aka / anonymous */) [C:\\Home\\CS\\TextAnalysis\\Ezhik\\node_modules\\mongodb\\lib\\url_parser.js:~7] [pc=000003CC6CBB279E] (this=000001186EB04381 ,url=0000004FD766F421 ) 2: arguments adaptor frame: 2->1 3: connect(aka connect) [C:\\Home\\CS\\TextAnalysis\\Ezhik\\node_modules\\mongodb\\lib\\mongo_client.js:~390] [pc=000...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

Answer 1

you cannot use forEach in this fashion with Promisified GetSuggestion call. Please use Promises to iterate though TextObject. See the below link(#mistake number 2) to understand how to use Promises for such scenarios.

Answer 2

Since you are receiving a "JavaScript heap out of memory" error, and since the relevant function works when called outside a loop, you are getting stuck in an infinite loop somehow. This is relatively easy to do with while loops, which you use in TextHandler.js .

I suspect your issue is one or more of the following:

1. DB.GetSuggestions isn't importing into TextHandler.js properly. Can you confirm that the function works within TextHandler.js when it is not inside an iterative loop?
2. The while loop should use == or === (try both) for comparison in its condition, not = .
3. In ImportText you create a new undefined variable result and in the next line use it in a comparison condition for a while loop. You probably don't intend for result to be undefined here, and it could be causing your infinite loop (assuming reg.exec(fullText)) is otherwise working correctly).

Let me know if these steps help.

Answer 3

 TextObject.Words.forEach(function(wd){ // iteration

    console.log(wd.WordText);
    DB.GetSuggestions(wd.WordText).then(function(suggestions){ // async block
      wd.Options = suggestions;
    });
  });

You are iterating over async block DB.GetSuggestions(wd.WordText) which is not waiting for async block to finish and moves on to next item in iteration. You need to you Promises to handle async functions with ease. You can your Promise.all .

I found answer from " Iterate over async function " for your question to be most relevant