filebeat/logstash convert json to csv format

Question

Need an advice and sample code as i am new to filebeat/logstash configuration. The harvester sends a dsv file to filebeat and the filebeat is taking the dsv input and sending an json output to logstash server.

Can I send the dsv file format to logstash and filter the input and give csv format? What changes to be done to filebeat and logstash to get csv output?

I am trying to convert the json input to csv in logstash but getting an empty file. We are using logstash 6.x version. Filebeat input dsv file looks like this:

dbname~dbdate~dbtime
db01~120-03-2019~08:15

Answer 1

If you can send db01~120-03-2019~08:15 to logstash via filebeat, that string can easily be parsed by using the digest filter or the csv filter (using ~ as a separator). And then you can use the csv output to write csv to a file.

Or you can use the mutate filter with gsub to replace the ~ with a comma in the message and then write the message to the file with the file output.