Why reinstalled Filebeat knows previous logs were already loaded to Elasticsearch

Question

I previously used Filebeat to load log data to the Elasticsearch though logstash, then I would like to try it again. So I reinstalled the Filebeat and emptied the Elasticsearch data, and then tried to reload the log data by Filebeat to Elasticsearch. But the Filebeat already knows that the data has been loaded once eventhough the Elasticsearch data storage is emptied. How Filebeat knows that log data was previously loaded? and if I would like to load again all log data, what should I do?

Answer 1

You need to clear the registry_file so that the "history" of read files is cleared as well.

To change the default configuration for the registry_file you just need to specify the full configuration path in the config file ( filebeat.yml ): https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#_registry_file

For example:

filebeat:
  registry_file: /var/lib/filebeat/registry