将日志发送到elasticsearch时logstash输出错误

Question

I read my services logs using filebeat ,then filebeat sends the logs to logstash for parsing. logstash sends the parsed logs to elasticsearch to be indexed. But today this error happens in logstash and no logs is indexed in elasticsearch.

[2021-11-02T11:35:36,774][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_inde ::Event:0xf85da17>], :response=>{"index"=>{"_index"=>"logstash-alias-000015", "_type"=>"_doc", "_id"=>"YNas33wBlcfHwocoMbSU", "status"=>400, "error"=>{"type" ls.Usage.UserUsage] of type [float] in document with id 'YNas33wBlcfHwocoMbSU'. Preview of field's value: 'NaN'", "caused_by"=>{"type"=>"illegal_argument_exc ]"}}}}}

I searched but did not find a clue, Any help is much appreciated.

Answer 1

According to the error you get, a field called ls.Usage.UserUsage of type float contains the value 'NaN' which is not a float.

What you could do is to modify your mapping to set the ignore_malformed setting to true so that this value is ignored, but it won't prevent the document from being indexed.

The other option is to make sure to not produce such wrong values upstream.