Logstash：輸出到Elasticsearch會導致Ruby出錯，但可與stdout一起使用

Question

錯誤與紅寶石代碼有關，我該如何解決？

配置：

input {
  beats {
    port => 5044
    ssl => false
  }
}

filter {
  if [type] == "apache" {
ruby {
code => " if event['message']
event['message'] = event['message'].gsub('\x','Xx')
event['message'] = event['message'].gsub('\x','XXx')
end
"
}

json {
      source => "message"
    }
}
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    sniffing => true
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

Logstash日志：

{：timestamp =>“ 2016-07-01T13：23：30.475000 + 0100”，：message =>“連接被拒絕”，：class =>“ Manticore :: SocketException”，：backtrace => [“ / opt / logstash / vendor / bundle / jruby / 1.9 / gems / manticore-0.6.0-java / lib / manticore / response.rb：37：in initialize'“，” org / jruby / RubyProc.java：281：incall'“，” / opt / logstash / vendor / bundle / jruby / 1.9 / gems / manticore-0.6.0-java / lib / manticore / response.rb：79：in call'“，” /opt/logstash/vendor/bundle/jruby/1.9 /gems/manticore-0.6.0-java/lib/manticore/response.rb:256:incall_once'“，” /opt/logstash/vendor/bundle/jruby/1.9/gems/manticore-0.6.0-java/lib /manticore/response.rb:153:in code'“，” /opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.18/lib/elasticsearch/transport/transport/http/manticore。 rb：84：inperform_request'“，” org / jruby / RubyProc.java：281：in call'“，” /opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.18/lib/ elasticsearch / transport / transport / base.rb：257：inperform_request'“，” /opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch -transport-1.0.18 / lib / elasticsearch / transport / transport / http / manticore.rb：67：in perform_request'“，” /opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0。 18 / lib / elasticsearch / transport / transport / sniffer.rb：32：inhosts'“，” org / jruby / ext / timeout / Timeout.java：147：in timeout'“，” / opt / logstash / vendor / bundle / jruby / 1.9 / gems / elasticsearch-transport-1.0.18 / lib / elasticsearch / transport / transport / sniffer.rb：31：inhosts'“，” /opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch- transport-1.0.18 / lib / elasticsearch / transport / transport / base.rb：79：in reload_connections！'“，” /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.7。 0-java / lib / logstash / outputs / elasticsearch / http_client.rb：72：insniff！'“，” /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.7.0-java /lib/logstash/outputs/elasticsearch/http_client.rb:60:in start_sniffing！'“，” org / jruby / ext / thread / Mutex.java：149：insynchronize'“，” / opt / logstash / vendor / bundle / jruby / 1.9 / gems / logstash-outp ut-elasticsearch-2.7.0-java / lib / logstash / outputs / elasticsearch / http_client.rb：60：in start_sniffing！'“，” org / jruby / RubyKernel.java：1479：inloop'“，” / opt / logstash /vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.7.0-java/lib/logstash/outputs/elasticsearch/http_client.rb:59:in'start_sniffing！'“]，：level =>：錯誤} {：timestamp =>“ 2016-07-01T13：23：52.470000 + 0100”，：message =>“ CircuitBreaker :: rescuing exceptions”，：name =>“ Beats input”，：exception => LogStash :: Inputs :: Beats :: InsertingToQueueTakeTooLong，：level =>：warn} {：timestamp =>“ 2016-07-01T13：23：52.470000 + 0100”，：message =>“ Beats輸入：斷路器已檢測到減速或停轉在管道中，輸入將關閉當前連接並拒絕新連接，直到管道恢復為止。“，：exception => LogStash :: Inputs :: BeatsSupport :: CircuitBreaker :: HalfOpenBreaker，：level =>：warn} {：timestamp =>“ 2016-07-01T13：23：52.471000 + 0100”，：message =>“ CircuitBreaker :: rescuing exceptions”，：name =>“節拍輸入”，：exception => LogStash： ：Inputs :: Beats :: InsertingToQueueTakeTooLong，：level =>：warn} {：timestamp =>“ 2016-07-01T13：23：52.471000 + 0100”，：message =>“ Beats輸入：斷路器已檢測到減速或停滯在管道中，則輸入將關閉當前連接並拒絕新連接，直到管道恢復為止。“，：exception => LogStash :: Inputs :: BeatsSupport :: CircuitBreaker :: HalfOpenBreaker，：level => :: warn} { ：timestamp =>“ 2016-07-01T13：23：53.471000 + 0100”，：message =>“節拍輸入：管道被阻塞，暫時拒絕新連接。”，：reconnect_backoff_sleep => 0.5，：level => ::警告} {：timestamp =>“ 2016-07-01T13：23：53.472000 + 0100”，：message =>“ CircuitBreaker :: Open”，：name =>“ Beats input”，：level =>：warn} {：timestamp =>“ 2016-07-01T13：23：53.473000 + 0100”，：message =>“節拍輸入：斷路器已檢測到管道中的減速或停頓，該輸入正在關閉當前連接並拒絕新連接，直到管道恢復。“，：exception => LogStash :: Inputs :: BeatsSupport :: CircuitBreaker： ：OpenBreaker，：level =>：warn} {：timestamp =>“ 2016-07-01T13：23：53.972000 + 0100”，：message =>“節拍輸入：管道被阻塞，暫時拒絕新連接。”，： reconnect_backoff_sleep => 0.5，：level =>：warn}

Answer 1

問題不是來自ruby代碼，而是來自您與Elasticsearch的連接。

在您的堆棧跟蹤中：

“ /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch -2.7.0- java / lib / logstash / outputs / elasticsearch / http_client.rb：59：in

您的Elasticsearch實例未啟動，或者logstash無法訪問該實例。

Answer 2

我通過刪除elasticsearch的索引解決了這個問題。

curl -XDELETE 'http://localhost:9200/filebeat-*'

請注意，紅寶石不起作用。 我不得不將日志格式更改為合並日志以處理異常。