简体   繁体   English

Node JS forEach内存泄漏问题

[英]Node JS forEach memory leak issue

I've been creating a small node js app that iterates through an array of names and queries an API for the names. 我一直在创建一个小型节点js应用程序,该应用程序迭代名称数组并查询API以获取名称。 The issue I have is that the array is very large (400,000+ words) and my application runs out of memory before the forEach is complete. 我的问题是数组非常大(400,000多个单词),并且在forEach完成之前我的应用程序内存不足。

I've been able to diagnose the issue by researching about how JS works with the call stack, web api, and callback queue. 通过研究JS如何与调用堆栈,Web API和回调队列一起使用,我已经能够诊断出该问题。 What I believe the issue to be is that the forEach loop is blocking the call stack and so the http requests continue to clog up the callback queue without getting resolved. 我认为要解决的问题是forEach循环阻塞了调用堆栈,因此http请求继续阻塞回调队列,而没有得到解决。

If anyone can provide a solution for unblocking the forEach loop or an alternative way of coding this app I would be very greatful. 如果任何人都可以提供解决方案来阻止forEach循环或对该应用进行编码的另一种方式,我将非常感激。

Node JS App Node JS应用

const mongoose = require("mongoose");
const fs = require("fs");
const ajax = require("./modules/ajax.js");

// Bring in Models
let Dictionary = require("./models/dictionary.js");


//=============================
//     MongoDB connection
//=============================

// Opens connection to database "test"
mongoose.connect("mongodb://localhost/bookCompanion");
let db = mongoose.connection;

// If database test encounters an error, output error to console.
db.on("error", (err)=>{
  console.error("Database connection failed.");
});

db.on("open", ()=>{
  console.info("Connected to MongoDB database...");
}).then(()=>{

  fs.readFile("./words-2.json", "utf8", (err, data)=>{

    if(err){
      console.log(err);
    } else {
      data = JSON.parse(data);

      data.forEach((word)=>{
      let search = ajax.get(`API url Here?=${word}`);
      search.then((response)=>{
        let newWord = new Dictionary ({
          word: response.word,
          phonetic: response.phonetic,
          meaning: response.meaning
        }).save();
        console.log("word saved");
      }).catch((err)=>{
        console.log("Word not found");
      });
      });

    };
  });

});

Check 校验

  1. Check whether api accepts multiple query params. 检查api是否接受多个查询参数。
  2. Try to use async Promises. 尝试使用异步Promises。
  3. Resolve the promises and try to perform the save operation on the Promises by Promise#all 兑现承诺,并尝试在Promise#all by Promise#allPromises上执行保存操作

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM