logstash不会将旧日志导入ES

Question

When I start logstash, the old logs are not imported into ES.
Only the new request logs are recorded in ES.
Now I've see this in the doc .

Even if I set the start_position=>"beginning" , old logs are not inserted.
This only happens when I run logstash on linux.

If I run it with the same config, old logs are imported.
I don't even need to set start_position=>"beginning" on windows.

Any idea about this ?

Answer 1

When you read an input log to Logstash, Logstash will keep an record about the position it read on this file, that's call sincedb .

Where to write the sincedb database (keeps track of the current position of monitored log files). 
The default will write sincedb files to some path matching "$HOME/.sincedb*"

So, if you want to import old log files, you must delete all the .sincedb* at your $HOME. Then, you need to set

start_position=>"beginning"

at your configuration file.

Hope this can help you.

Answer 2

Please see this line also.

This option only modifies "first contact" situations where a file is new and not seen before. If a file has already been seen before, this option has no effect.