Use input ElasticSearch document fields in Watcher fields

Question

I have the following watch set up which triggers when it sees Error-level messages. It is triggering correctly and my watch history shows this. The idea is that we send a SOAP message to our monitoring software which uses it to format an email to the right person.

The problem is that I assumed I could get the information for the first record in the input using mustache commands like {{ctx.payload.hits.hits.0.fields.Environment}} but any place where I use it to get the document's fields it is blank instead. I can query the payload to get the number of records found as expected though which is the weird part.

My watch is as follows:

{
  "trigger": {
    "schedule": {
      "interval": "60s"
    }
  },
  "input": {
    "search": {
      "request": {
        "indices": ["<logstash-{now/d{YYYY.MM.dd}}-cat>"],
        "body": {
          "query": {
              "bool": {
                  "must": [
                {
                  "match": {
                    "Level": "Error"
                  } 
                },
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-60s",
                      "lt": "now"
                    }
                  }
                }
              ]
              }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "gt": 0
      }
    }
  },
  "throttle_period": "30m",
  "actions": {
        "log_error_patrol_webhook": {
            "webhook": {
                "method": "POST",
                "host": "myhost",
                "port": 9080,
                "path": "/path/",
                "headers": {
                    "Content-Type": "text/xml"
                },
                "body": "<soapenv:Envelope xmlns:soapenv=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:imap=\"http://blueprint.bmc.com/ImapiElems\" xmlns:bas=\"http://blueprint.bmc.com/BasicTypes\" xmlns:even=\"http://blueprint.bmc.com/Event\"><soapenv:Header/><soapenv:Body><imap:SendEvent><imap:connection>1111111</imap:connection><imap:message><!--Zero or more repetitions:--><bas:NameValue_element><bas:name>msg</bas:name><bas:value><bas:string_value>Environment {{ctx.payload.hits.hits.0.fields.Environment}}. Found {{ctx.payload.hits.total}} errors in the logs.</bas:string_value></bas:value></bas:NameValue_element><bas:NameValue_element><bas:name>mc_host</bas:name><bas:value><bas:string_value>{{ctx.payload.hits.hits.0.fields.HostName}}</bas:string_value></bas:value></bas:NameValue_element><bas:NameValue_element><bas:name>mc_host_class</bas:name><bas:value><bas:string_value>{{ctx.payload.hits.hits.0.fields.Environment}}</bas:string_value></bas:value></bas:NameValue_element><bas:NameValue_element><bas:name>mc_object_class</bas:name><bas:value><bas:string_value>App: {{ctx.payload.hits.hits.0.fields.appindex}}</bas:string_value></bas:value></bas:NameValue_element><bas:NameValue_element><bas:name>mc_object</bas:name><bas:value><bas:string_value>https://test.kibana.net/app/kibana#/doc/[logstash-]YYYY.MM.DD[-{{ctx.payload.hits.hits.0.fields.appindex}}]/{{ctx.payload.hits.hits.0.fields._index}}/http?id={{ctx.payload.hits.hits.0.fields._id}}</bas:string_value></bas:value></bas:NameValue_element><bas:NameValue_element><bas:name>severity</bas:name><bas:value><bas:string_value>CRITICAL</bas:string_value></bas:value></bas:NameValue_element><bas:NameValue_element><bas:name>bw_notification</bas:name><bas:value><bas:string_value>email@host</bas:string_value></bas:value></bas:NameValue_element><even:subject>Logstash</even:subject></imap:message><imap:timeout>3000</imap:timeout><imap:messageClass>LOGSTASH</imap:messageClass><imap:messageType>MSG_TYPE_NEW_EVENT</imap:messageType></imap:SendEvent></soapenv:Body></soapenv:Envelope>"
            }
        }
    }
}

And the email it returns looks like this:

Incident Time:   Wednesday, 18 May 2016 16:06:14 +0800
          Host:   
      Severity:   CRITICAL
  Object Class:   
        Object:   https://test.kibana.bwainet.net/app/kibana#/doc/[logstash-]YYYY.MM.DD[-]//http?id=
     Parameter:   
      Location:   Unknown
       Message:   Environment . Found 4 errors in the logs

I've been crawling over the elastic documentation as to why this wouldn't be working but I've come up at a loss. Any clue why these fields are blank for me when the fields certainly exist in our elastic documents? I'm hoping it's just a syntax error

Cheers guys

Answer 1

In your Mustache calls use something like this ctx.payload.hits.hits.0._source.Environment . Basically, every ctx.payload.hits.hits.0.fields. needs to be replaced with ctx.payload.hits.hits.0._source. , except _id and _index .

For _index you need ctx.payload.hits.hits.0._index and for _id you need ctx.payload.hits.hits.0._id .