使用Pickle vs数据库加载大量数据？

Question

I have previously saved a dictionary which maps image_name -> list of feature vectors, with the file being ~32 Gb. I have been using cPickle to load the dictionary in, but since I only have 8 GB of RAM, this process takes forever. Someone suggested using a database to store all the info, and reading from that, but would that be a faster/better solution than reading a file from disk? Why?

Answer 1

Use a database because it allows you to query faster. I've done this before. I would suggest against using cPickle. What specific implementation are you using?