MongoDB使用未知的findOne方法導致大延遲 - New Relic

Question

我設置newrelic以更好地了解我的應用程序有什么瓶頸，我發現了一個我似乎無法弄清楚的問題。

我的大部分延遲都是由mongoDB user.fineOne引起的，但主要問題是我似乎無法找到代碼中的位置。

在下圖中，您可以看到調用我的API的get/all/proposal端點的跟蹤詳細信息。 它首先是14個方法調用，它們是我的server.js中的中間件，之后它是一個中間件：認證它有MongoDB用戶findOne ，那就是延遲。

獲取/所有/提案的代碼：

app.get('/all/proposals',isLoggedIn,function(req, res) {
   Proposal.find().sort({proposalNo: -1}).limit(5).exec(function(err,proposal){
       if(err){
           console.log(err);
       }else{
           console.log("All Proposals " + proposal);
           res.json(proposal);
       }
   });
});

現在，我無法看到我在get/all/proposals上運行MongoDB上的User.findOne調用 。 最初我認為是isLoggedIn中間件，我檢查用戶是否在會話中（Passport.js）但是你可以看到isLoggedIn Middleware只需要0.0222（ms）。

同樣的問題出現在多個API端點上，即get/caseStudy ，它始終是user.findOne下面是另一個例子：

誰能幫我解決這個問題。 如果您需要更多詳細信息，請告訴我。

更新： Server.js代碼

 // set up ======================================================================

require('newrelic');
var express  = require('express');
var app      = express();                               // create our app w/ express
var server = require('http').createServer(app);
var mongoose = require('mongoose');                     // mongoose for mongodb
var port     = process.env.PORT || 8080;                // set the port
var database = require('./config/db');                  // load the database config
var morgan = require('morgan');                         // log requests to the console (express4)
var bodyParser = require('body-parser');                // pull information from HTML POST (express4)
var methodOverride = require('method-override');        // simulate DELETE and PUT (express4)
var passport = require('passport');
var flash    = require('connect-flash');
var session      = require('express-session');
var cookieParser = require('cookie-parser');
var compression = require('compression');
var nodemailer = require('nodemailer');
var busboy = require("connect-busboy");

// configuration ===============================================================


mongoose.connect(database.url);                                 // connect to mongoDB database on modulus.io
require('./config/passport')(passport);
app.use(express.static(__dirname + '/public'));  
app.use(express.static(__dirname + '/views'));  // set the static files location /public/img will be /img for users
app.use(busboy());
app.use(compression()); //use compression
app.use(morgan('dev'));                                         // log every request to the console
app.use(bodyParser.urlencoded({'extended': true}));             // parse application/x-www-form-urlencoded
app.use(bodyParser.json());                                     // parse application/json
app.use(bodyParser.json({ type: 'application/vnd.api+json' })); // parse application/vnd.api+json as json
app.use(methodOverride());
app.use(cookieParser());                                        // read cookies (needed for auth)
app.set('view engine', 'ejs');                                  // set up ejs for templating

// required for passport
app.use(session({ secret: '',  resave: false, saveUninitialized: false })); // session secret
app.use(passport.initialize());
app.use(passport.session());                                    // persistent login sessions
app.use(flash());                                               // use connect-flash for flash messages stored in session

 // routes ======================================================================

require('./routes/index.js')(app, passport); // load our routes and pass in our app and fully configured passport
//require('./routes/knowledgeBase/index.js')(app, passport);
require('./routes/bios/index.js')(app, passport);

 // listen (start app with node server.js) ======================================

app.listen(port);
console.log("App listening on port " + port);

更新2： Passport.js

// config/passport.js

// load all the things we need
var LocalStrategy   = require('passport-local').Strategy;
var crypto = require("crypto");
var api_key = '';
var domain = '';
var mailgun = require('mailgun-js')({apiKey: api_key, domain: domain});
// load up the user model
var User            = require('../app/models/user');

 // expose this function to our app using module.exports
 module.exports = function(passport) {

// =========================================================================
// passport session setup ==================================================
// =========================================================================
// required for persistent login sessions
// passport needs ability to serialize and unserialize users out of session

// used to serialize the user for the session
passport.serializeUser(function(user, done) {
    done(null, user.id);
});

// used to deserialize the user
passport.deserializeUser(function(id, done) {
    User.findById(id, function(err, user) {
        done(err, user);
    });
});

// =========================================================================
// LOCAL SIGNUP ============================================================
// =========================================================================
// we are using named strategies since we have one for login and one for signup
// by default, if there was no name, it would just be called 'local'

passport.use('local-signup', new LocalStrategy({
        // by default, local strategy uses username and password, we will override with email
        firstNameField: 'firstName',
        lastNameField: 'lastName',
        usernameField: 'email',
        passwordField: 'password',
        jobTitleField: 'jobTitle',
        startDateField: 'startDate',
        passReqToCallback: true // allows us to pass back the entire request to the callback
    },

    function(req, email, password, done) {

        // find a user whose email is the same as the forms email
        // we are checking to see if the user trying to login already exists
        User.findOne({
            'email': email
        }, function(err, user) {
            // if there are any errors, return the error
            if (err)
                return done(err);

            // check to see if theres already a user with that email
            if (user) {
                return done(null, false, {
                    message: 'That email is already taken.'
                });
            }
            else {

                var token = crypto.randomBytes().toString();
                // if there is no user with that email
                // create the user
                var newUser = new User();

                // set the user's local credentials
                newUser.firstName = req.body.firstName;
                newUser.lastName = req.body.lastName;
                newUser.email = email;
                newUser.password = newUser.generateHash(password); // use the generateHash function in our user model
                newUser.jobTitle = req.body.jobTitle;
                newUser.startDate = req.body.startDate;
                newUser.birthday = req.body.birthday;
                newUser.region = req.body.region;
                newUser.sector = req.body.sector;
                newUser.accountConfirmationToken = token;
                newUser.accountConfirmationTokenExpires = Date.now() + 3600000;
                newUser.accountVerified = 'false';



                // save the user
                newUser.save(function(err) {
                    if (err)
                        throw err;
                    else {
                        return done(null, newUser);
                    }
                });
            }

        });

    }));

// =========================================================================
// LOCAL LOGIN =============================================================
// =========================================================================
// we are using named strategies since we have one for login and one for signup
// by default, if there was no name, it would just be called 'local'

passport.use('local-login', new LocalStrategy({
    // by default, local strategy uses username and password, we will override with email
    usernameField : 'email',
    passwordField : 'password',
    passReqToCallback : true // allows us to pass back the entire request to the callback
},
function(req, email, password, done) { // callback with email and password from our form

    // find a user whose email is the same as the forms email
    // we are checking to see if the user trying to login already exists
    User.findOne({ 'email' :  email }, function(err, user) {
        // if there are any errors, return the error before anything else
        if (err)
            return done(err);

        // if no user is found, return the message
        if (!user)
            return done(null, false, req.flash('loginMessage', 'No user found.')); // req.flash is the way to set flashdata using connect-flash

        // if the user is found but the password is wrong
        if (!user.validPassword(password))
            return done(null, false, req.flash('loginMessage', 'Oops! Wrong password.')); // create the loginMessage and save it to session as flashdata

        if(user.accountVerified == 'false')    
            return done(null, false, req.flash('loginMessage', 'Looks like you have not verified your account after registeration.'));
        else
            user.lastLogin = Date.now();
            user.save(function(err) {
                if (err)
                    throw err;
                else {
        // all is well, return successful user
                    return done(null, user);
                }
            });
    });

}));

};

更新3： isLoggedIn函數

 // route middleware to make sure a user is logged in
  function isLoggedIn(req, res, next) {

// if user is authenticated in the session, carry on 
if (req.isAuthenticated())
    return next();

    // if they aren't redirect them to the home page
    res.redirect('/');
 }

更新4：獲取提案的步驟

第1步：首先加載提案頁面

 app.get('/proposals',isLoggedIn,function(req, res) {
    res.render('proposals.ejs', {
        user : req.user // get the user out of session and pass to template
    });
});

第2步：提議頁面有一個angular.js控制器/工廠，它在頁面加載時調用以下函數來獲取數據。

// =========================================================================
// FUNCTIONS TO BE RUN WHEN THE PAGE FIRST LOADS TO POPULATE FRONT-END =====
// =========================================================================
$scope.intialize = function() {
    $scope.getAllSectors();
    $scope.getLatestProposals();
}

// ===============================
// GET LATEST *5* PROPOSALS  =====
// ===============================
factory.getLatestProposals = function() {

    return $http.get('/all/proposals')
        .then(function(response) {
            //promise is fulfilled
            deferred.resolve(response.data);

            console.log("readched the filtered project service!");

            //promise is returned
            // return deferred.promise;
            return response.data;

        }, function(response) {
            deferred.reject(response);

            //promise is returned
            return deferred.promise;
        });
};

第3步：調用/all/proposals路由

 // =======================
//  GET All Proposals =====
//  =======================
app.get('/all/proposals',isLoggedIn,function(req, res) {
   Proposal.find().sort({proposalNo: -1}).limit(5).exec(function(err,proposal){
       if(err){
           console.log(err);
       }else{
           console.log("All Proposals " + proposal);
           res.json(proposal);
       }
   });
});

Answer 1

查看您提供的代碼后，性能日志中顯示的.findOne()似乎是在搜索用戶並對其進行身份驗證時執行的。

因此，似乎性能瓶頸發生在以下2個查詢之一：

/*
 * LOCAL LOGIN
 */
// find a user whose email is the same as the forms email
    // we are checking to see if the user trying to login already exists
    User.findOne({ 'email' :  email }, function(err, user) {
...


/*
 * LOCAL SIGNUP
 */
    // find a user whose email is the same as the forms email
    // we are checking to see if the user trying to login already exists
    User.findOne({
        'email': email
...

我看到您在兩個護照本地策略中搜索email字段，因此您可以通過在該字段上創建索引來提高性能。

要嘗試優化LOCAL LOGIN findOne查詢，您可以嘗試在email字段中為users集合添加索引（如果您還沒有）：

// This index will optimize queries that search against the {email} field
db.users.createIndex({ email: 1});

更新＃1

我發現了一個相關的Stack Overflow主題可能是您性能問題的答案 - 您應該更新express.js配置中的以下行：

app.use(session({ secret: '',  resave: false, saveUninitialized: false }));

至

app.use(session({ secret: '',  resave: true, saveUninitialized: true }));

我還設法找到關於這些筆記resave並saveUninitalized快遞JS文件的屬性：

saveUninitialized

強制將“未初始化”的會話保存到商店。 會話在新的但未修改時未初始化。 選擇false對於實現登錄會話，減少服務器存儲使用或遵守設置cookie之前需要許可的法律非常有用。 選擇false也有助於客戶在沒有會話的情況下發出多個並行請求的競爭條件。

默認值為true，但不推薦使用默認值，因為默認值將來會更改。 請研究此設置並選擇適合您的用例的內容。

請注意，如果您將Session與PassportJS結合使用，Passport將向會話添加一個空的Passport對象，以便在用戶通過身份驗證后使用，這將被視為對會話的修改，從而導致其被保存。 這已在PassportJS 0.3.0中修復

重新保存

強制會話保存回會話存儲，即使會話在請求期間從未被修改過。 根據您的商店，這可能是必要的， 但它也可能創建競爭條件，其中客戶端向您的服務器發出兩個並行請求，並且在另一個請求結束時對一個請求中的會話所做的更改可能會被覆蓋，即使它沒有進行任何更改 （此行為還取決於您正在使用的商店）。

默認值為true，但不推薦使用默認值，因為默認值將來會更改。 請研究此設置並選擇適合您的用例的內容。 通常，你會想要虛假。

我怎么知道這對我的商店是否有必要？ 最好的方法是檢查商店是否實施了觸摸方法。 如果是，那么您可以安全地設置resave：false。 如果它沒有實現觸摸方法，並且您的商店在存儲的會話上設置了到期日期，那么您可能需要重新保存：true。