内连接查询太慢了

Question

I'm writing a query that is supposed to retrieve words from "Words table" if they are contained in sentences in the "Sentences table"

For example: the query should output "hello" if it finds at least one sentence that contains the word "hello"

I was able to write this query so far :

SELECT DISTINCT (words.word) FROM sentences inner join words on sentences.sentence LIKE CONCAT('% ', words.word ,' %')

The issue with this query that it's super slow, like it took 8hours+ and did not output any results given that the words table is around 250k rows and the sentence table is around 1M rows. Can anyone help with a faster solution.

Answer 1

With the information you shared, the data is not extremely large, but the 'sentence' table is going thru full-tables-can for each distinct word in 'words' table.
Also distinct has a processing span involved.
It would be prudent to index the words & sentence table
( probably words on initiating Letter & Sectence on same) & partition them too based on same criteria. \

Then run same query as above.

This approach might reduce how much data being joined per key match.