What's the most efficient way to load a dictionary in Python?

Question

I have what is a Python dictionary of ~1000 entries. A script is going to be called repeatedly that will want to parse a string and see if any keys in the string match. If they do, it will take some action based on the key and the value.

Which of these is faster?

1) Store the dictionary in a MySQL database, and then read the database each time the script is called?

2) Store the dictionary in a Python script and import it every time? (eg make a file that contains nothing but the dictionary initialization)

3) Store the dictionary in a text file, and import it every time? (either a flat text file or a pickle serialized data file, using cpickle)

Just looking for a best practice here.

Answer 1

You might create a .py Python file that just assigns the dictionary to a name. Save the file. Compile the file to a .pyc then load it as a module when needed by your main Python script.

You get the advantage of keeping a readable textual representation of your dict for maintenance/debug, the speed of loading a .pyc file and the simplicity of it all being standard Python.

Answer 2

出于测试目的，您还可以加载带有任何内容的字典（在本例中为整数），如下所示：

    D = dict(zip(range(100),range(100)))

Answer 3

I would think that storing it as a dictionary in a python file and importing into every module that needs it would be the way to go. Can you construct it programmatically? Either way, the file will only be actually imported once per program execution so it shouldn't be a big deal unless you know that loading it once at the beginning is unacceptable for some reason.

shelve could be another way to go here. This would probably be the way to do it if you wanted to go with option (3) It's built on the anydbm module. This will probably be slower but will allow you to avoid having the whole thing in memory at once.

In my opinion, both 1) and 3) are right out. The overhead of making the database queries would, in all probability, slow access down dramatically. Option 2) will make it all a simple dict lookup.