我们如何在没有logstash和beats的情况下通过java将数据摄取到弹性搜索

Question

How can we ingest data to elastic search through java without logstash and beats is there any option like kafka or something like using only java without any tools

Answer 1

I am not sure why you dont want to consider Filebeats --> Elastic. But yes, there are other ways to send your logs to Elastic search.

Also, you did not mention whats the source, whether you want to insert app logs, database. Assuming you want to send microservices logs also, and below options holds good for sending other data too.

As you dont want to use Filebeat, you should add custom code to collect, refine, format and publish the logs.

• you can use Kafka Sink Connector to Elastic search to move all your logs
• Also, you can use UDP protocol to send(client) logs and listen(server), then implement buffer and ingest to Elastic.

you can develop a commons lib which holds all this code and use in all your java applications.

Simple udp client server code - https://github.com/suren03/udp-server-client