用水槽或kafka替换掉

Question

I need your help with one suggestion. In current scenario, we have one application on cloud and via splunk we have the ability to view log. I am thinking of implementing this using our big data tools like flume/kafka wherein I can take real time log data from cloud ( currently taken by splunk ) and made it available to our HDFS. Few concern here

1. is this feasible and make sense ?
2. for log search (same capability like splunk )
   

which tool can we use?

Answer 1

If you just want to move logs into HDFS, you can use Flume with HDFS sink.
There are also few other options available like -

Logstash

You can use other framework like Elasticsearch and Kibana to have more functionality available for the logs.