How can I create an alert in OMS when a Linux service is stopped?

Question

我正在尝试在Linux服务停止时在OMS中创建警报。

Answer 1

AFAIK we have below options to accomplish your requirement.

Option I :

If the service/deamon is configured by default configurations then the service log information would be logged under /var/log/messages.

Whenever a Linux service is stopped if the information is getting logged in /var/log/messages file then follow below steps to get alerted:

1. Goto Azure portal -> YOURLOGANALYTICSWORKSPACE -> Advanced settings -> Data -> Syslog -> type 'daemon' -> click '+' -> click 'save'. For more information, refer this https://docs.microsoft.com/en-us/azure/azure-monitor/platform/data-sources-syslog link.

2. Goto Azure portal -> YOURLOGANALYTICSWORKSPACE -> Logs -> type 'Syslog' -> click 'Run'. Check 'SyslogMessage' column in the output. Output also have various other useful columns like SeverityLevel, ProcessName and ProcessID which you may use while developing the query based on your need.

3. So query would look something like shown below.

   Syslog | where (Facility == "daemon") | where (SyslogMessage has "xxxxxxx" and SyslogMessage has "stopping") | summarize AggregatedValue= any(SyslogMessage) by Computer, bin(TimeGenerated, 30s)

4. Create and configure custom log alert in the Log Analytics workspace alert tile by using above query. Set the threshold value, frequency, period details while configuring an alert. Provide intended action group to get notified on alert getting triggered.

Option II :

If the service/deamon is custom configured then the service log information would be logged in that particular custom path.

Whenever a Linux service is stopped if the information is getting logged in /xxxx/yyyy/zzzz.txt file (or other examples are /aaaa/bbbb/jenkins/jenkins.log, cccc/dddd/tomcat/catalina.out, etc.) then follow below steps to get alerted:

1. Goto Azure portal -> YOURLOGANALYTICSWORKSPACE -> Advanced settings -> Data -> Custom Logs -> click 'Add +' -> .... For more information, please refer this https://docs.microsoft.com/en-us/azure/azure-monitor/platform/data-sources-custom-logs link.

2. Goto Azure portal -> YOURLOGANALYTICSWORKSPACE -> Logs -> type 'CUSTOMLOGNAME_CL' -> click 'Run'. Check something like 'RawData' column in the output.

3. So query would look something like shown below.

   CUSTOMLOGNAME_CL | where (RawData has "xxxxxxx" and RawData has "stopping") | summarize AggregatedValue= any(RawData) by Computer, bin(TimeGenerated, 30s)

4. Create and configure custom log alert in the Log Analytics workspace alert tile by using above query. Set the threshold value, frequency, period details while configuring an alert. Provide intended action group to get notified on alert getting triggered.

Option III :

In case your service log data can't be collected with custom logs also then send the data directly to Azure monitor using HTTP Data Collector API that is explained here -> https://docs.microsoft.com/en-us/azure/azure-monitor/platform/data-collector-api .

An example using runbooks in Azure Automation is provided in Collect log data in Azure Monitor with an Azure Automation runbook is explained here -> https://docs.microsoft.com/en-us/azure/azure-monitor/platform/runbook-datacollect .

Hope this helps!! Cheers!! :)