[英]ELK Stack - Remote servers' logs parsed and classed correctly, but dumped to stout not elasticsearch
正如(冗长的)主题所建议的,我的远程服务器将日志发送到logstash服务器。
这里是logstash配置:
输入:
[root@tool01 conf.d]# cat 01-lumberjack-input.conf
input {
lumberjack {
port => 5000
type => "logs"
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}
}
类型:
[root@tool01 conf.d]# cat 10-syslog.conf
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
输出:
[root@tool01 conf.d]# cat 30-lumberjack-output.conf
output {
elasticsearch { host => localhost }
stdout { codec => rubydebug }
}
本地日志按指示正常工作,但是来自远程的所有日志都发送到stdout:
[root@tool01 logstash]# tail -n 19 logstash.stdout
{
"message" => "Nov 29 20:16:44 foreman dhcpd: DHCPACK on 192.168.50.100 to 3c:4a:92:12:1c:cb via eth0",
"@version" => "1",
"@timestamp" => "2014-11-29T20:16:44.000Z",
"type" => "syslog",
"file" => "/var/log/messages",
"host" => "foreman.ics.dmz",
"offset" => "3511785",
"syslog_timestamp" => "Nov 29 20:16:44",
"syslog_hostname" => "foreman",
"syslog_program" => "dhcpd",
"syslog_message" => "DHCPACK on 192.168.50.100 to 3c:4a:92:12:1c:cb via eth0",
"received_at" => "2014-11-29 07:16:46 UTC",
"received_from" => "foreman.ics.dmz",
"syslog_severity_code" => 5,
"syslog_facility_code" => 1,
"syslog_facility" => "user-level",
"syslog_severity" => "notice"
}
这可能是一个愚蠢的问题,但是我只在客户端服务器上安装了托运人,我是否还需要在其中运行logstash?
提前致谢!
事情比我最初预期的要混乱。
例如,此日志:
Dec 2 10:39:12 foreman dhcpd: DHCPACK on 192.168.50.52 to 6c:ad:f8:26:b1:68 (Chromecast) via eth0
图形下方的列表视图正确列出了时间,但条形图的条目位于23:39:12?
13小时的延迟听起来令人难以置信,但是秒到13小时听起来更像是时区问题? 然而,所有运行的日期都与按日期验证的date
相同
更多信息:
[root@tool01 tmp]# curl -s XGET http://192.168.50.241:9200/logstash-2014.12.03/_search?pretty=true | tail -n 15; TZ=UTC date
{
"_index": "logstash-2014.12.03",
"_type": "syslog",
"_id": "bOVXPpf9SFOKKPgx7PWfCA",
"_score": 1.0,
"_source": {
"message": "Dec 3 00:06:30 foreman dhcpd: DHCPREQUEST for 192.168.50.251 from 00:15:5d:32:14:09 via eth0",
"@version": "1",
"@timestamp": "2014-12-03T00:06:30.000Z",
"type": "syslog",
"file": "/var/log/messages",
"host": "foreman.ics.dmz",
"offset": "3756888",
"syslog_timestamp": "Dec 3 00:06:30",
"syslog_hostname": "foreman",
"syslog_program": "dhcpd",
"syslog_message": "DHCPREQUEST for 192.168.50.251 from 00:15:5d:32:14:09 via eth0",
"received_at": "2014-12-02 11:06:31 UTC",
"received_from": "foreman.ics.dmz",
"syslog_severity_code": 5,
"syslog_facility_code": 1,
"syslog_facility": "user-level",
"syslog_severity": "notice"
}
},
{
"_index": "logstash-2014.12.03",
"_type": "syslog",
"_id": "5axCp7UgRxmunclFqlgKNw",
"_score": 1.0,
"_source": {
"message": "Dec 3 00:06:30 foreman dhcpd: DHCPACK on 192.168.50.251 to 00:15:5d:32:14:09 via eth0",
"@version": "1",
"@timestamp": "2014-12-03T00:06:30.000Z",
"type": "syslog",
"file": "/var/log/messages",
"host": "foreman.ics.dmz",
"offset": "3756982",
"syslog_timestamp": "Dec 3 00:06:30",
"syslog_hostname": "foreman",
"syslog_program": "dhcpd",
"syslog_message": "DHCPACK on 192.168.50.251 to 00:15:5d:32:14:09 via eth0",
"received_at": "2014-12-02 11:06:31 UTC",
"received_from": "foreman.ics.dmz",
"syslog_severity_code": 5,
"syslog_facility_code": 1,
"syslog_facility": "user-level",
"syslog_severity": "notice"
}
},
{
"_index": "logstash-2014.12.03",
"_type": "syslog",
"_id": "m_MuAZPcS8ixmCn5getvyg",
"_score": 1.0,
"_source": {
"message": "Dec 3 00:06:30 spacewalk dhclient[906]: DHCPREQUEST on eth0 to 192.168.50.240 port 67 (xid=0x421c37e1)",
"@version": "1",
"@timestamp": "2014-12-03T00:06:30.000Z",
"type": "syslog",
"file": "/var/log/messages",
"host": "spacewalk.ics.dmz",
"offset": "269907",
"syslog_timestamp": "Dec 3 00:06:30",
"syslog_hostname": "spacewalk",
"syslog_program": "dhclient",
"syslog_pid": "906",
"syslog_message": "DHCPREQUEST on eth0 to 192.168.50.240 port 67 (xid=0x421c37e1)",
"received_at": "2014-12-02 11:06:33 UTC",
"received_from": "spacewalk.ics.dmz",
"syslog_severity_code": 5,
"syslog_facility_code": 1,
"syslog_facility": "user-level",
"syslog_severity": "notice"
}
}
Tue Dec 2 19:19:14 UTC 2014
如此看来,数据在写入EL之前已被缓冲?
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.