dictionary interface for large data sets

Question

I have a set of key/values (all text) that is too large to load in memory at once. I would like to interact with this data via a Python dictionary-like interface.

Does such a module already exist?

Reading key values should be efficient and values compressed on disk to save space.

Edit:

• Ideally cross platform, but only using Linux for now
• Needs to be thread safe

Answer 1

Take a look at the shelve module. It provides dictionaries with various backing database systems include BerkleyDB and dbm.

Answer 2

尝试bsddb（Berkley db的Python绑定）

Answer 3

最终基于此现有模块创建了自己的解决方案 。

Answer 4

假设您使用的是* nix平台，那么您只是乞求tokyo-cabinet ...它具有一组本机的ruby绑定...您可以在1978th.net上获取更多信息...