Filebeat with Redis

Question

Base on elastic documentation on filebeat, plugins provided to insert events into redis via filebeat, but it`s under Redis list(file structure type).

https://www.elastic.co/guide/en/beats/filebeat/current/redis-output.html

Just wonder if I want to insert all events in Hashes, is that possible ?

Regards, SIMON

Answer 1

The reason for this is that the Redis list data type acts as a temporary queue that contains your data before it is consumed by some other component (logstash, etc). On the other side of the pipeline, the consuming component will simply call LPOP/BLPOP to remove the first (oldest) element from the list and process it.

A Redis hash data type is not usable as a queue, you need to ask yourself how the consumer component would be able to consume data that is stored in hashes. For instance, the Logstash redis input plugin allows to use either the list , channel or pattern_channel data type, all of which can act as a temporary queue from which consumers can get and process data.