Reason to upgrade to elastic search to 5.x from 2.x

Question

Currently i have a system running elastic search 2.4 with 40-50 indices holding very little data (may be few hundreds of documents) used for data logging. There is a ask to add additional feature to index upto 50 billion documents. Now as this is new requirement, there is also thinking to update to 5.x but need solid data points as why this cannot be done on 2.4x.

1. Firstly community support available for 2.x? As 5.x is latest, would users of issues be encouraged to move to 5.x rather than solving in 2.x?
2. Are there major changes in architectural wise leading to performance improvements on a) heap b) CPU consumption c) Anything else? d) support for spark or hadoop for batch processing

Answer 1

Yes. Many reasons. Even if you don't care about support , Performance enhancements in 5.x is good reason to migrate.

ES 5 uses newer version of Lucene in addition to several improvements in ES code itself.

Also since you have very few documents now - this is the right time to upgrade ; It will be much easier / time saving than upgrading a larger index.