Unknown notfication is really struggling me and the official docs seems newer than released bosun binary, like unknownPost
is ilegal option.
The following unknown template and notification do not work. I can see unknown notfications in bosun UI, but it doesn't send notification(http alert). Other normal alerts work fine.
template tmpl_unknown {
body = 'x'
subject = `{{$v := .Eval .Alert.Vars.val }}{"warn_undo": [{"app":"xxx","db_type":"NOSQL","db_cat":"aaaa","ip":"{{.Group.host}}","warn_level":1,"warn_type":"cpu","warn_detail":"cpu {{.Group.device}} total util is {{$v.Value | pct}} ","notifier": "xxxxxxxx","status":"undo","report_time":""}]}`
# body = {{.Name}}: {{.Group | len}} unknown alerts
# subject = `{"warn_undo": [{"app":"aaaa","db_type":"NOSQL","db_cat":"aaaa","ip":"x.x.x.x","warn_level":1,"warn_type":"alive","warn_detail":"Time: {{.Time}} xName: {{.Name}} Alerts:{{range .Group}}{{.}},{{end}}","notifier": "xxxxxxxx","status":"undo","report_time":""}]}`
unknown_alive = `
{"warn_undo": [{"app":"aaaa","db_type":"NOSQL","db_cat":"aaaa","ip":"x.x.x.x","warn_level":1,"warn_type":"alive","warn_detail":"Time: {{.Time}} Name: {{.Name}} Alerts:{{range .Group}}{{.}},{{end}}","notifier": "xxxxxxxx","status":"undo","report_time":""}]}`
}
unknownTemplate = tmpl_unknown
notification gcsx_unknown {
post = ${sys.gcsAlert}
contentType = application/json
print = True
timeout = 5m
next = gcsx_unknown
unknownBody = unknown_alive
bodyTemplate = subject
# unknownMinGroupSize = 5
# unknownThreshold = 0
}
alert cpu_total_util_or_unknown {
template = tmpl_unknown
$val = avg(q("none:0all-last:cpu.pct_used{host=*,device=total}", "1m", ""))
warn = $val > 60
warnNotification = gcsx_unknown
runEvery = 1
unknown = 1m
}
I want to alert cpu usage. If host is gone, no data reported and I will got an unknown alert.
Bosun version: the newest compiled from master branch(2018.10.08)
After digging the code, I found the issue finally and it's simply:
unknown notification must be sent as critNotification
cmd/bosun/sched/check.go:290
incident.NeedAck = true
switch event.Status {
case models.StCritical, models.StUnknown:
notify(a.CritNotification)
case models.StWarning:
notify(a.WarnNotification)
}
Change warn
, warnNotification
to crit
and critNotification
and notfication is sent successfully. This should be documented in bosun.org.
unknownPost
is a legal option, just like unknownPost
, unknownGet
, unknownBody
, unknownEmailSubject
. But they should be defined in template variables, or it will report error:
2018/10/11 16:19:36 fatal: main.go:130: couldn't read rules: conf: bosun_rule.conf:47:0: at <alert x...>:
notification x uses template key unknown_post in unknown post url, but template cpu_util does not include it
The right way is:
template tmpl_unknown {
unknown_alive = `
{"warn_undo": [{"app":"xxx","db_type":"NOSQL","db_cat":"aaaa","ip":"x.x.x.x","warn_level":1,"warn_type":"alive","warn_detail":"Time: {{.Time}} Name: {{.Name}} Alerts:{{range .Group}}{{.}},{{end}}","notifier": "xxx","status":"undo","report_time":""}]}`
unknown_post = "//x.x.x.x:7777/alert"
# unknown_post = {{ V "$gcsAlert" }}
}
notification gcsx_unknown {
post = ${sys.gcsAlert}
contentType = application/json
print = True
timeout = 5m
next = gcsx_unknown
unknownBody = unknown_alive
unknownPost = unknown_post
}
As you note that the http url prefix http:
need to be removed.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.