苹果系统日志到ELK

Question

Anyone know how to export the ASL log of mac os x in a SIEM like ELK ? Is there a configuration on filebeat to transform them first in a text format.

I know we can read them via syslog –T utc –F raw

Answer 1

You should be a able to export everything via the log command. I sometimes like to analyze the logs my personal laptop produces through logstash and elasticsearch. To get the logs into logstash I run the following in a tmux window:

sudo log stream --info --debug |sudo -E socat -dddd STDIN TCP4:logstash.docker:6661,interval=4,reuseaddr,forever

I hacked together a couple of groks in logstash to parse the macos system logs and they work most of the time. Feel free to use them for your own logs. If you manage to improve them please send them (or a link to the gist) my way.

filter {
  grok {
    match => { "message" => "%{YEAR:year}-%{MONTHNUM:month}-%{MONTHDAY:day}\s+%{HOUR:hour}:%{MINUTE:minute}:%{SECOND:second}%{ISO8601_TIMEZONE:tz}%{GREEDYDATA}" }
    add_field => {"[log_timestamp]" => "%{year}-%{month}-%{day} %{hour}:%{minute}:%{second}%{tz}"}
    tag_on_failure => ["datefail_string"]
  }
  grok {
    match => { "message" => ["%{YEAR:year}-%{MONTHNUM:month}-%{MONTHDAY:day}\s+%{HOUR:hour}:%{MINUTE:minute}:%{SECOND:second}%{ISO8601_TIMEZONE:tz}\s+%{BASE16NUM:thread_id}\s+%{LOGLEVEL:log_type}\s+%{BASE16NUM:activity}\s+%{NUMBER:pid}\s+%{DATA:program}:\s+\(%{DATA:library}\)\s+\[%{DATA:package}\]\s+%{GREEDYDATA:msg}",  "%{YEAR:year}-%{MONTHNUM:month}-%{MONTHDAY:day}\s+%{HOUR:hour}:%{MINUTE:minute}:%{SECOND:second}%{ISO8601_TIMEZONE:tz}\s+%{BASE16NUM:thread_id}\s+%{LOGLEVEL:log_type}\s+%{BASE16NUM:activity}\s+%{NUMBER:pid}\s+%{DATA:program}:\s+\[%{DATA:package}\]\s+%{GREEDYDATA:msg}", "%{YEAR:year}-%{MONTHNUM:month}-%{MONTHDAY:day}\s+%{HOUR:hour}:%{MINUTE:minute}:%{SECOND:second}%{ISO8601_TIMEZONE:tz}\s+%{BASE16NUM:thread_id}\s+%{LOGLEVEL:log_type}\s+%{BASE16NUM:activity}\s+%{NUMBER:pid}\s+%{DATA:program}:%{GREEDYDATA:msg}"] }
      tag_on_failure => ["logfail1"]
    remove_field => ["message"]
  }

  mutate {
    remove_field => ["month","day","year","minute","hour","second","tz"]
  }
}

Answer 2

You can use a log collector agent like NXLog EE to send logs directly to Elasticsearch, or forward collected logs to Logstash and MacOS is a supported platform. Kernel logs, local Syslog, Apple System Logs, process accounting logs, and other log files can be collected. EE is not free but there is a trial available.

There are also modules to parse Apple System Log (*.asl) files.