Export data from Elasticsearch to CSV using Logstash

Question

How can I export data from Elasticsearch to CSV using Logstash? I need to include only specific columns.

Answer 1

Install 2 plugins : elasticsearch input plugin and csv output plugin . Then create a configuration file . Here is a good example for this particular case .

You are ready to go now, just run:
bin/logstash -f /path/to/logstash-es-to-csv-example.conf

And check export.csv file specified in output -> csv -> path .

Answer 2

Important note: There is a known bug in csv output plugin when working with Logstash 5.x. The plugin generates a string of %{host} %{message}%{host} %{message}%{host} %{message} . There's an open issue for it: https://github.com/logstash-plugins/logstash-output-csv/issues/10

As a workaround you may:

• downgrade to Logstash 2.x until this gets resolved

• use the file output instead

   file { codec => line { format => "%{field1},%{field2}"} path => "/path/to/data_export.csv" } 

• modify the plugin's code according to the github discussion...