努力寻找 memory 泄漏。 NodeJS，快递，MongoDB

Question

so I have a NodeJS with Express configured for API calls to get data from my backend Database.

Everything works great except there is a memory leak that I can't seem to solve no matter what I do. I have narrowed it down to a MongoDB fetch call, that grabs data from MongoDB.

Since the Data being sent is the same, and to avoid multiple requests to MongoDB, I created a top level variable that fetches that data so on a request it sends that, vs doing a fetch constantly as it would be 1000's of requests every minute.

I have also set the --max_old_space_size variable to 4096 and higher at 8192 and will eventually crash also.

Below is the fetch code.

//router get apis.js

 import { Model } from '../../dbmodels/models.js';

 let data = null;

// Bot interval
 setInterval(async () => {
     try {
         data = await tokenModel.find({
             $or: [
                 { "currentRanks.minuteTokenRank": {$lt: 51} },
                 { "currentRanks.fiveMinuteTokenRank": {$lt: 51} },
                 { "currentRanks.fifteenMinuteTokenRank": {$lt: 51} },
                 { "currentRanks.thirtyMinuteTokenRank": {$lt: 51} },
                 { "currentRanks.hourlyTokenRank": {$lt: 51} },
                 { "currentRanks.dailyTokenRank": {$lt: 51} },
                 { "currentRanks.weeklyTokenRank": {$lt: 51} }
             ]
         }).lean();
     } catch (error) {
       console.error(error);
       return;
     }
   }, 45000);

export async function main(req, res) {
    let dataRes = data;
    try {
        res.status(200).json(dataRes);
        dataRes = null;
    } catch {(err) => {
        res.status(500).json({ message: err.message })
        console.log('err', err.message)
        }
    }
  //console.log('Get Top Data')
}

//main server.js file

import dotenv  from "dotenv"
dotenv.config()

import express from 'express';
const app = express();

import { createServer } from 'https';
import { createServer as _createServer } from 'http';

import { readFileSync } from 'fs';

import compression from "compression";

import pkg from 'mongoose';
const { connect, connection } = pkg;
import cors from 'cors';
import auth from './utils/auth.js'

connect(process.env.DB_URL);
let mongoClient = connection;
mongoClient.on('error', (error) => console.error(error));
mongoClient.once('open', () => console.log(`Cncted to DB ${mongoClient.name}`));

app.use(compression());
app.use(cors({ origin: ['http://localhost:3000']}));

// Apis route

import apis from './apis/apis.js';
app.use('/api', auth, apis);

//listen both http & https
const httpServer = _createServer(app);
const httpsServer = createServer({
    key: readFileSync('redacted.pem'),
    cert: readFileSync('redacted.pem'),
}, app);

httpServer.listen(3000, () => console.log('Server Started port 3000'));

httpsServer.listen(3001, () => {
    console.log('HTTPS Server running port 3001')
})

Answer 1

So looks like I was able to find the leak. It wasn't with any of the API's I posted. But a hidden one that I use or Web3. There is a known bug on the web3 package leaving connections open.

This is tied to the data I am retrieving above which is why it seemed to me like it was this API, but further troubleshooting found the real issue.

Here is the issue in case anyone uses web3 packages and runs into similar.

https://github.com/web3/web3.js/issues/3042#issuecomment-663622882