有关将SQL Server数据移至NoSQL或ES / Solr的建议

Question

I support a Webforms application using a SQL server db.

We have a Find functionality in the app, which essentially reads from the db, based on the filter criteria selected in the Find page. I would like make this more performant and dynamic, similar to a search engine.

I tried moving the data from SQL Server to ES but was not successful. I also need the data to be moved to the NoSQL db or search engine in real time.

Is it possible for ES / Solr or NoSQL to directly read from SQL Server ? Would that cause latency?

Apologies if the above description is vague but I am initially trying to get a high-level understanding of the possible options.

Thanks in advance.

Answer 1

So what you want (and correct me if I'm mistaken) is to transport all the data from your SQL server into Solr?

If that's the case then you can use a JDBC and a managed schema to do so. A JDBC essentially lets you connect Solr to a SQL database, do a query and index the results. This can be used with Solr's data import handler, however the DIH is kinda crap and not suitable for production environments (though acceptable for a one off index), see this link . I've had issues with it just not working and being a general turd.

A JDBC isn't Solr specific, it's just a Java program that connects to a database, performs a query and receives the results. It's not hard to make your own that would take the SQL results, format them into a Solr friendly format and post them to your core/collection. Here's an intro to JDBC's and there's plenty more out there.

A managed schema is just a flexible schema that infers your field types and creates them as needed, good for a first pass until you need to fine tune the schema. So as the SQL results are indexed, Solr will just create a field type for each column.

So with a bit of coding and configuration you can get it like so:

1. Create Solr core/collection with default schema and config
2. Create JDBC to connect to your DB and perform necessary query
3. Have JDBC convert SQL results to Solr friendly fashion (there should be info out there about that too, it's not too hard, Solr accpets many formats)
4. Make it post said results to your core/collection
5. Run JDBC