Dictionary searching words - best performance in PHP and MySQL

Question

I would like to implement a dictionary for searching words and their inflection. Dictionary size is about 60MB and it is stored in txt file.

Currently I'm passing a whole dictionary (60MB) to an variable and then searching for interesting words. I can't resist an impression that is not properer way (to allocate 60MB in memory just for dictionary)

Can anyone recommend other solution? Maybe there are some other ways to search for interesting phrases within a file without storing whole content of it into memory? or maybe putting whole dictionary into MySQL db would help (I could slice that dictionary and put every word into db with additional id to recognize inflection between different word that are coming form the same root)

Answer 1

Of course putting dictionary inside a DB would help. Database works MUCH faster due to internal indexing and residing in system memory as a daemon; Searching inflections would also be simpler, faster and more efficient with queries:

SELECT * FROM dictionary WHERE word LIKE "devel%"

would get you "developing", "developer", "develop" and so on. Also there's a lot of other possibilities. The only drawback is that you must have database server installed. On the other hand, loading an entire txt file in memory would consume more then 1GB of memory if 20 users connect at the same time, since script is started anew each time.

Answer 2

Putting 60mb in memory to search is ludicrous....a database is the only right answer here. I answered a similar question about dictionaries yesterday here .

Once you have it in a database, also look into fulltext search as answered here .