Logstash正在将旧数据放入Elasticsearch。 无法清除Elasticsearch数据

Question

I have Logstash reading from a log file status.log and sending the output to an Elasticsearch instance.

I want to clean the data in Elasticsearch, for that I'm executing curl -XDELETE 'http://localhost:9200/index_name/_all' . If I check through the head plugin, the data is gone.

If that was not enough I'm also cleaning the log file with echo "" > status.log .

When I execute the application again, the old data reappears in Elasticsearch but with an updated @timestamp . The data is not present again in the status.log . The new data is inserted correctly in Elasticsearch.

How can I get rid of the old data? Is it still stored in Elasticsearch or Logstash has some kind of cache?

Answer 1

assuming you are working with the file plugin. If you add the stdout plugin to your output section like so:

stdout { codec => rubydebug }

logstash will display each processed log element on the console. When using the file plugin every processed logmessage gets a path field telling you were logstash read the message from. maybe that helps you to find out where the messages are coming from...