Reading a file from a URI in logstash

Question

I have a file sitting on a remote server. I know the file by its URI:

http://some.server/dir/derp/syslog.log

When I use the logstash file input, the URI is rejected with an error complaining of a relative path. There is no security on the remote file. I can curl it without issue from the logstash system.

My newb-config input snippet looks like:

input {
        file {
        path => "http://some.server/dir/derp/syslog.log"
        type => "syslog"
        }

What's the right way to read the file?

Answer 1

The file input plugin doesn't read remote files, it only allows to read files from the local host.

You have a few solutions:

A. You can install filebeat on your remote server some.server to tail that syslog file and either ship it to your logstash or directly to your ES server.

B. You can install Logstash directly on your remote server and have the file input plugin tail that file directly on the server.

C. You can use the http_poller input plugin in order to retrieve that file via HTTP.

input {
  http_poller {
    urls => {
      mysyslog => "http://some.server/dir/derp/syslog.log"
    }
    request_timeout => 60
    interval => 60
  }
}