Python 3：存储数据而不将其加载到内存中

Question

I am currently building a flask app that will have lots of data that I think I cannot load into memory. I have searched many places, and have found the SQL seems to be a good solution. Sadly I cannot use SQL for this project due to some limitations of SQL.

My project consists many entries of

database[lookupkey1][lookupkey2]...and more lookup keys

My current plan is to override __getitem__ and __setitem__ and __delitem__ and replace them with calls to the database. Is their any kind of database that can store large amounts of maps/dictionaries like

{"spam":{"bar":["foo","test"],"foo":["bar"]}}

I am also currently using JSON to save data, so it would be appreciated if the database had a easy way to migrate my current database.

Sorry that I'm not very good at writing stack overflow questions.

Answer 1

Most document-oriented DBs like MongoDB would allow you to save data as nested dict-list-like objects and query them using their keys and indexes.

PS Accessing such a DB through a Python's dict accessors is a bad idea as it would produce a redundant DB query for each step which is highly ineffective and may lead to performance problems. Try looking at ORM for a DB you choose as most ORMs would allow you to access document-oriented DB's data in a way similar to accessing dicts and lists.