Spring Boot application with current elasticsearch version

Question

I've recently tried to include elasticsearch into one of my projects. Therefore I looked up the different possibilities to do so. It basically came down to use either a) the transport client (internal protocol cluster nodes too use to communicate = my understanding) or b) implement it all as REST HTTP calls on my own.

As b) doesn't look like a good option, and I'm used to spring data anyways I tried using spring-data-elasticsearch which was updated recently to version 3.0.0 to support es 5.5.0 which is great as the last version was very old.

Well, I tried everything but didn't manage to get it to work with my project, as this really seems to be some kind of dependency hell you get into. My project is too big, I just can't upgrade it to spring boot 2.0.0M4 which I believe would be required to support the new spring-data-elasticsearch version - right?

So, could somebody please advice on the current and preferred way for now and the future is on how to deal with elasticsearch in spring projects? Do we really have to implement the whole REST API ourselves? Am I missing something?

Here is the REST client which should be used, if there is one used, right?

This is the compatibility chart I know which is old.

Thanks!

Answer 1

Unfortunately there are quite a few options out there:

• There's the transport client you have mentioned. While not (yet) deprecated, it's not the way forward and I would not start a new project with it (if possible).
• There is the low level REST client (mentioned in the blog post you've linked), which was added in 5.0, but is compatible with older versions as well. While it works, you don't really want to use it, since it's pretty low level (as the name suggests).
• The high level REST client has just been released with 5.6 — this is what you want to use going forward (and it will decouple you from a specific Spring Boot / Data version). Right now it only supports the index, delete, aggregate, search, and bulk APIs, but for regular operations this should be enough. More APIs will be added in the future and you can always fall back to the low level client if you need to do other tasks (like setting up a specific mapping for an integration test).
• With the release of Spring Data Kay, it also supports Elasticsearch 5.x. If you need to use Spring Data, this is what you'll want to use, but keep in mind that it's a community project and development can be slower than everybody wants (though they do accept pull requests ;-) ). As you've discovered, you'll need to use Spring Boot 2 to have Spring Data Kay built in. This might be a good reason to upgrade or it might be a blocker for you.

Answer 2

How about writing a small service which would be a standalone application in spring boot 2, with the newest ES dependency, and giving it a REST API compatible with your requirements for the rest of the project? You could communicate with it via RestTemplate for starters.

This way you can start breaking up your large project into smaller subprojects and eventually maybe even microservices. Also spring boot 2 has some really nice reactive options, which really come in handy for processing data.