如何索引 Elasticsearch 的标准化 SQL 数据库

Question

I have already indexed a database (SQL) with a single table which is in sync with its Elasticsearch index. Now I want to index a database with multiple normalized tables. So, how should I index those tables? Should I write multiple JOIN queries in my logstash file during indexing the database tables, or should I index each table one by one and perform multiple index search? But for second way, I do not know how to form elasticsearch query for the relevant SQL queries. I am new to Elasticsearch. So any guidance for the problem would be appreciated. Here I am also attaching the schema of the database. One more thing, I am using PHP client for searching and displaying data.

Answer 1

First of all, everything will depend on how you want to build your indexes in elasticsearch, that is, if you want an index for each table or an index for several tables.
My advice is:

1. Create a trigger in the database to audit every change (insert, update, delete) and store it in a news table along with the action and a state.
2. Create a view for each type of novelty or table, this view will depend on how you want to index everything.
3. Use jdbc to call the views says the state is equal to pending (raw).
4. Use filters to normalize your data and adjust it to your elasticsearch structure.
5. Use JDBC output to update the database by setting the newness to processed to prevent it from appearing in the query.

In addition to these points, my recommendation is that you have those tables in a single index, for example, employee, where you can create different nested objects for each entity in the database, such as department, etc. where you could add the code tags and description of each one. You can check if it has not been clear