How to monitor Python scripts using Sensu?

Question

I would like to use Sensu Core to monitor python scripts and I am confused how to do it.

From Sensu documentation this requires Sensu Checks . In the provided example ruby script checks that chef-client is running:

#!/usr/bin/env ruby

# get the current list of processes
processes = `ps aux`

# determine if the chef-client process is running
running = processes.lines.detect do |process|
  process.include?('chef-client')
end

# return appropriate check output and exit status code
if running
  puts 'OK - Chef client process is running'
  exit 0
else
  puts 'WARNING - Chef client process is NOT running'
  exit 1
end

How to implement such a check for a specific script and not the application? That is, how would I go about monitoring a specific python script (eg test.py) and not python in general?

Answer 1

So, I have been running some python scripts in sensu for my AWS Linux clients successfully , this is a good example of my check definition:

{
 "checks": {
"check-rds-limit": {
  "interval": 86400, 
  "command": "/etc/sensu/plugins/rds-limit-check.py",
  "contacts": [
    "cloud-ops"
  ],
  "occurrences": 1,   
  "subscribers": [
    "cloud-ops-subscription"
  ], 
  "handlers": [
    "email",
    "slack"
  ]
}
  }
}

And your python plugin can start with defining the shebang path:

#!/usr/bin/env python
import sys
...
...
//<code condition for warning>
sys.exit(1)
//<code condition for critical>
sys.exit(2)
//<code condition where everything is fine>
sys.exit(0)

Answer 2

More generally the above script is searching for the string chef-client in the running processes. You could replace that with any other string, like test.py , which would detect if any program running has test.py in its name. (You might need to match the sub-string test.py if you run the program with python test.py , I don't know ruby.)

I would suggest that you use Sensu process check plugin for a more general function which includes more customization. Look at the other sensu plugins too.

Answer 3

Why not monitor the expected result or operation of the script rather than the process itself. Typically, we will setup checks to monitor an end point, in the case of a web application, or an observed behavior, such as messages in a database, to determine if the application is running.

There will be times when the process is technically running but not anything due to an error condition or resource issue. Monitoring the expected result is a much better option than watching a process.