Elasticsearch - 从 2.x 升级到 7.x - 使用 TransportClient 和 Jest 进行双重写入

Question

We are in process of migrating from elasticsearch 2.x to 7.x, the approach is to dual write to both the clusters. Later in time, restore the snapshot from 2.x to 7.x in multiple hops of 2.x to 5.x then 5.x to 6.x and finally 6.x to 7.x.

Now, the issue is enabling dual write to old and new clusters, we are using TransportClient (tcp connection) and Jest (http connection) to index documents, is there any client which can handle this? As far as I know, Jest doesn't have support for 7.x, what are our choices here?

EDIT

We are using Kafka connect to ship the same updates to 7.x elasticsearch. This question is still open for people looking for non-kafka solutions.

Answer 1

You are right JEST doesn't support the 7.X and Elasticsearch official clients are not available for 2.X :) and till now there is no client which can write to both the versions, even Elasticsearch clients are released with its server version and are not compatible when there is a change in the major version.

Also, As you are using Java, I would highly recommend using the high-level client , instead of transport the client. As mentioned in official ES blog , soon transport clients would be deprecated in favor of the high-level clients. text from the same blog:

The high-level client will eventually replace the transport client in the future, hence we invite all our Java users to try it out and migrate to it if possible. The current release supports the following Elasticsearch APIs:

Solution:

You need to use maven shade plugin to relocate the class exist in either of your jest or Elasticsearch client as they have conflicting classes and this way you can use both the clients and using them you can index and query their Elasticsearch servers.

More info on shade plugins can be found here and here .