Nodejs在生产中调试错误

Question

I have a nodejs script running in production

I unlikely (once in thousand times) get errors like this:

TypeError: value is out of bounds
    at checkInt (buffer.js:1009:11)
    at Buffer.writeUInt16LE (buffer.js:1067:5)
    at Object.foo.bar (/fake/path/name.js:123:1);
    at Object.foo.bar2 (/fake/path/name2.js:123:1);
    at Object.foo.bar3 (/fake/path/name3.js:123:1);

Causing the production server to crash...

Great I have a stacktrace! But I want to know what it's current data is for each call or all the data?

What are some great tools or code to use to for error logging(with it's current data) on production code?

Answer 1

I highly recommend in using either Winston or Bunyan . The selection of npm package is a decision of your application.

You can benchmark the available npm packages by going through the stats in the available in npm pages. The stats are basically the following.

1. downloads in the last day
2. downloads in the last week
3. downloads in the last month
4. open issues and open pull requests.

Having higher number of downloads recently will indicate that there is a great support for the module you are using in the long run. So that is important.

Both Winstan and Bunyan being best logging npm packages in the market, the main difference is that, Winstan is really awesome and flexible for normal logging purposes. Of course Winstan offers a great deal of logging capabilities. But how ever, to make use of these capabilities, some effort need to put on compared to Bunyan.

Bunyan on the other hand is specially support the fact of " analysing logs ". So basically Bunyan is for logs processing. So if you want to analyse your logs, log files it is highly recommend to use Bunyan. Tweaking logs with Bunyan is fairly easy vis-a-vis to Winstan.

I did a thorough comparison between Bunyan and Winstan. Please do check the link below to view how the Winstan and Bunyan can use depending on the scope, use-case and necessity of the logging to the Node application. link : https://docs.google.com/document/d/1pD9PLyxlcHVxxOvserNLO9tAz-QA_Co-xo6cWLhLghc/edit?usp=sharing

Also in the Production environment , make sure to use the logging levels wisely. The mostly used logging levels in production environment are:

• error
• info
• debug

Answer 2

Yoou can use Winston or Pino

With winston you can load many modules for log where you want and maybe store logs online. I never use pino, but i have read good things about it.

Set env variables for choose where output your log, for example you wont to show output on stdout only if you are in develop and store online only if the app is in production.

Answer 3

A good way to handle the asynchronous functions in node.js is by using the decofun debug tool .

The main feature of it is to parse the code and the names of the anonymous functions according to their context.

You can deanonymise any anonymous function by running it with deco filename.js

A simple example would be as mentioned in the documentation

function gravy() {
   return function returnedﾠfromﾠgravyﾠㅣlineﾠ2 () {
       return {
         prop: function asﾠpropertyﾠpropﾠㅣlineﾠ4 () {
           setTimeout(function passedﾠintoﾠsetTimeoutﾠㅣlineﾠ5 () {
             console.trace('Getting a trace...');  
           }, 10)

         }
       }
   }
}
Trace: Getting a trace...
    at passedﾠintoﾠsetTimeoutﾠㅣlineﾠ5 [as _onTimeout] (/home/ubuntu/workspace/node_modules/decofun/examples/loadable/index.js:6:22)
    at Timer.listOnTimeout (timers.js:92:15)

Since it comes with embedded cute-stack library which normalises the path to the current directory

By applying the command deco examples/loadable --cute table the output would be shown as

The best thing i like about it is that it transforms the functions based upon their calls to the original one as seen in the example from this

   function one (a, cb) {

}


one('blah', function () {

})

function two () {
  return function () { }
}


function three () {
  return {
    shoe: function () {}
  }
}

function four () {
  return function () { 
    return function () {

    }
  }
}

function five () {
  return function () {
    return function () {
      return function () {
        foo('blue', function () {

        })
      }
    }
  }
}


var six = function () {

}


var seven = function (err, cb) {

  return function () {
    cb(function () {

    })
  }

}

var o = {};
o.eight = function (cb) { }


o.eight(function () { })

o.eight.nine = function () {}
o.eight.nine(function () { })

var o2;

o2 = function () { }


;(function () {}())

!function () { }()



function toodeep () {
  return function () {
    return function () {
      return function () {

        return function () {
          return function () {
            return function () {

              return function () {
                return function () {
                  return function () {

                    return function () {

                    }                     

                  }
                }
              } 

            }
          }
        }        
      }
    }
  }
}

into this

function one (a, cb) {

}


one('blah', function passedﾠintoﾠoneﾠㅣlineﾠ6 () {

})

function two () {
  return function returnedﾠfromﾠtwoﾠㅣlineﾠ11 () { }
}


function three () {
  return {
    shoe: function asﾠpropertyﾠshoeﾠㅣlineﾠ17 () {}
  }
}

function four () {
  return function returnedﾠfromﾠfourﾠㅣlineﾠ22 () { 
    return function returnedﾠfromﾠᐸﾠreturnedﾠfromﾠfourﾠᐳﾠㅣlineﾠ23 () {

    }
  }
}

function five () {
  return function returnedﾠfromﾠfiveﾠㅣlineﾠ30 () {
    return function returnedﾠfromﾠᐸﾠreturnedﾠfromﾠfiveﾠᐳﾠㅣlineﾠ31 () {
      return function returnedﾠfromﾠᐸﾠreturnedﾠfromﾠᐸﾠreturnedﾠfromﾠfiveﾠᐳﾠᐳﾠㅣlineﾠ32 () {
        foo('blue', function passedﾠintoﾠfooﾠㅣlineﾠ33 () {

        })
      }
    }
  }
}


var six = function asﾠvarﾠsixﾠㅣlineﾠ42 () {

}


var seven = function asﾠvarﾠsevenﾠㅣlineﾠ47 (err, cb) {

  return function returnedﾠfromﾠᐸﾠasﾠvarﾠsevenﾠᐳﾠㅣlineﾠ49 () {
    cb(function passedﾠintoﾠcbﾠㅣlineﾠ50 () {

    })
  }

}

var o = {};
o.eight = function asﾠpropertyﾠeightﾠㅣlineﾠ58 (cb) { }


o.eight(function passedﾠintoﾠoːeightﾠㅣlineﾠ61 () { })

o.eight.nine = function asﾠpropertyﾠnineﾠㅣlineﾠ63 () {}
o.eight.nine(function passedﾠintoﾠeightːnineﾠㅣlineﾠ64 () { })

var o2;

o2 = function asﾠvarﾠo2ﾠㅣlineﾠ68 () { }


;(function IIFEﾠㅣlineﾠ71 () {}())

!function IIFEﾠㅣlineﾠ73 () { }()



function toodeep () {
  return function returnedﾠfromﾠtoodeepﾠㅣlineﾠ78 () {
    return function returnedﾠfromﾠᐸﾠreturnedﾠfromﾠtoodeepﾠᐳﾠㅣlineﾠ79 () {
      return function returnedﾠfromﾠᐸﾠreturnedﾠfromﾠᐸﾠreturnedﾠfromﾠtoodeepﾠᐳﾠᐳﾠㅣlineﾠ80 () {

        return function returnedﾠfromﾠᐸﾠreturnedﾠfromﾠᐸﾠreturnedﾠfromﾠᐸﾠreturnedﾠfromﾠtoodeepﾠᐳﾠᐳﾠᐳﾠㅣlineﾠ82 () {
          return function returnedﾠfromﾠᐸﾠreturnedﾠfromﾠᐸﾠreturnedﾠfromﾠᐸﾠreturnedﾠfromﾠᐸﾠreturnedﾠfromﾠtoodeepﾠᐳﾠᐳﾠᐳﾠᐳﾠㅣlineﾠ83 () {
            return function returnedﾠfromﾠᐸﾠreturnedﾠfromﾠᐸﾠreturnedﾠfromﾠᐸﾠreturnedﾠfromﾠᐸﾠreturnedﾠfromﾠᐸﾠreturnedﾠfromﾠtoodeepﾠᐳﾠᐳﾠᐳﾠᐳﾠᐳﾠㅣlineﾠ84 () {

              return function returnedﾠfromﾠᐸﾠreturnedﾠfromﾠᐸﾠreturnedﾠfromﾠᐸﾠreturnedﾠfromﾠᐸﾠreturnedﾠfromﾠᐸﾠreturnedﾠfromﾠᐸﾠreturnedﾠfromﾠtoodeepﾠᐳﾠᐳﾠᐳﾠᐳﾠᐳﾠᐳﾠㅣlineﾠ86 () {
                return function () {
                  return function () {

                    return function () {

                    }                     

                  }
                }
              } 

            }
          }
        }        
      }
    }
  }
}

Hope this might help a bit! Cheers!

Answer 4

I am using pm2 which is a process manager for node.js and also rollbar error reporting. I think you should define some metrics for the part of your code which this error comes from.

Answer 5

For any uncaughtException the server will stop in order to make the server keep on running even when there is an uncaught exception what i have done is created a separate collection for storing error, save error once an uncaught exception occurs and returns.

Collection

var ErrorSchema = new mongoose.Schema({
  err_Message:{type:String},
  err_Stack:{type:String},
  date:{type:Date}
});

Controller

process.on('uncaughtException', function (err) {
    console.log(err);
    console.error((new Date).toUTCString() + ' uncaughtException:', err.message);
    console.error(err.stack);

    var newError = new Error;
    newError.err_Message = err.message;
    newError.err_Stack = err.stack;
    newError.date = moment();
    newError.save(function(saveErr,errData){
        if(!saveErr)
            console.log('New Error is saved');
        else
            console.log('Error in saving error');
    });
    //process.exit(1)
});

The above methods stores the uncaught exception in the Error collection and the process/server does not stops.

Hope this helps.