用于聚合/报告数据的 Elasticsearch 与 RDMBs

Question

Has anyone has experience switching between Elasticsearch and a relational DB like mysql/postgres/? What are the pros/cons of both?

Background: looking to build a dashboard UI to show store/item related metrics and need the correct tool on the backend side that provides flexibility in queries (Imagine that the UI has selectors for date ranges and then the UI shows top items sold, total sales, etc.) in different time based charts. Some other notes are that we are just going to be using aggregations/nested aggregations (wouldn't be taking advantage of text search) around stores or items.

I know you could use both but which one is preferable in terms of

1. performance? I imagine that they would be largely similar
2. durability? I imagine elasticsearch and it automatically replicates data
3. maintenance? I imagine elasticsearch would be worse (maintaining a cluster vs maintaining a single node)
4. cost? I imagine an elasticsearch cluster storing the same amount of data would cost more because of replication
5. development work? I imagine elasticsearch would cause development to take longer using elasticsearch's custom queries vs writing APIs around sql queries

Are these assumptions correct? Are there other dbs/data stores that I should consider over these 2 options?

Answer 1

Based on my experience Elastic Search is a superb tool for :

1. Search
2. Real-time data Aggregation
3. Real-time reporting with extensive filtering support

We are also using Elastic Search for powering our real-time reports having extensive filter options (like date-range, status, etc).

We compared aggregation performance of ES and MongoDB with similar set of machines and for aggregating 5 million records mongo-db took around 12 Sec while ES took time under 1 sec.

performance? I imagine that they would be largely similar

If you have pure aggregation use case on loads of data requiring extensive filtering, searching etc then the performance of ES would be unmatched.

durability? I imagine elastic search and it automatically replicates data

Yes ES do have inherent replication support, as it is a distributed system.

maintenance? I imagine elasticsearch would be worse (maintaining a cluster vs maintaining a single node)

Definitely distributed systems demand more maintenance but you can use the Hosted version of ES (eg AWS Elasti-cache) as well

cost? I imagine an elasticsearch cluster storing the same amount of data would cost more because of replication

Considering cluster is required with replication support as well. Infra cost will be larger.

development work? I imagine elasticsearch would cause development to take longer using elasticsearch's custom queries vs writing APIs around sql queries

It depends on the experience with ES Since Mysql has been around for long, most dev folks are skilled with that. Any new technology has it's learning curve.

Keep in mind :

1. ES is not an ACID compliant datastore.
2. No Transactions support is there. If your system is purely transactional, then you may require relational-db as a read/write store and ES for powering aggregation use cases.