![](/img/trans.png)
[英]Ship filebeat logs to logstash to index with docker metadata
[英]Unable to index with docker logstash
我正在使用git@github.com:deviantony/docker-elk.git
存儲庫的最新代碼通過docker-compose up
命令托管 ELK 堆棧。 彈性搜索和 kibana 運行良好。
雖然我無法使用我的 logstash.conf 索引到 logstash,如下所示:
input {
file {
# Configure your path below
path => ["C:/Users/matt/Desktop/temp/logs/*.txt*"]
ignore_older => "141 days"
start_position => "beginning"
file_sort_by => "last_modified"
file_sort_direction => "desc"
sincedb_path => "NUL"
type => "appl"
codec => multiline {
pattern => "^<log4j:event"
negate => true
what => "previous"
}
}
}
filter {
if [type] == "appl" {
grok {
add_tag => [ "groked" ]
match => ["message", ".*"]
remove_tag => ["_grokparsefailure"]
}
mutate {
gsub => ["message", "log4j:", ""]
}
xml {
source => "message"
remove_namespaces => true
target => "log4jevent"
xpath => [ "//event/@timestamp", "timestamp" ]
xpath => [ "//event/@level", "loglevel" ]
xpath => [ "/event/message/text()", "message" ]
xpath => [ "/event/throwable/text()", "exception" ]
xpath => [ "//event/properties/data[@name='log4jmachinename']/@value", "machinename" ]
xpath => [ "//event/properties/data[@name='log4japp']/@value", "app" ]
xpath => [ "//event/properties/data[@name='log4net:UserName']/@value", "username" ]
xpath => [ "//event/properties/data[@name='log4net:Identity']/@value", "identity" ]
xpath => [ "//event/properties/data[@name='log4net:HostName']/@value", "hostname" ]
}
mutate {
remove_field => ["type"]
gsub => [
"message", "&", "&",
"message", "<", "<",
"message", ">", ">",
"message", """, "\"",
"message", "'", "'"
]
}
date {
match => [ "[timestamp][0]","UNIX_MS" ]
target => "@timestamp"
remove_field => ["timestamp"]
}
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "log4jevents"
user => "elastic"
password => "changeme"
ecs_compatibility => disabled
}
stdout {
codec => rubydebug
}
}
我想用我的logstash索引的日志文件如下所示
<log4j:event logger="Microsoft.Unity.ApplicationBlocks.Logging.Logger" timestamp="1615025506621" level="DEBUG" thread="13"><log4j:message>SSO->AccountController->Login->Before ClientID Check</log4j:message><log4j:properties><log4j:data name="log4jmachinename" value="hostname01" /><log4j:data name="log4japp" value="/LM/W3SVC/2/ROOT-1-132594985694777790" /><log4j:data name="log4net:UserName" value="IIS APPPOOL\default" /><log4j:data name="log4net:Identity" value="" /><log4j:data name="log4net:HostName" value="hostname01" /></log4j:properties><log4j:locationInfo class="Microsoft.Unity.ApplicationBlocks.Logging.Logger" method="Debug" file="F:\somefolder\Agent\_work\1\s\Unity\Microsoft.Unity.ApplicationBlocks\Logging\Logging.cs" line="353" /></log4j:event>
啟動docker-compose up
顯示的問題如下所示,用於 logstash
Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elastic:xxxxxx@localhost:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://elastic:xxxxxx@localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)"}
相同的 logstash.conf 在早期的 EK 版本 6.8 中工作。 我的 logstash.conf 有什么問題?
在您的 output elasticsearch
插件中,將hosts
屬性設置為elasticsearch:9200
。
output {
elasticsearch {
hosts => ["elasticsearch:9200"]
index => "log4jevents"
user => "elastic"
password => "changeme"
ecs_compatibility => disabled
}
stdout {
codec => rubydebug
}
}
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.