f.read的Python内存错误

Question

I have problem with memory in python "Python memory error". In fact , I try to recover data from a big .bson file using this script:

with open('xxxx.bson','rb') as f:
    data = bson.decode_all(f.read())

error message :

data = bson.decode_all(f.read())
MemoryError

thank you for any help you can provide

Answer 1

您可以通过切换到decode_file_iter来减少内存消耗，其中1）需要文件（不是其内容）作为输入，2）返回生成器。

Answer 2

I use this library: https://github.com/bauman/python-bson-streaming

from bsonstream import KeyValueBSONInput
f = open("xxxx.bson", 'rb')
stream = KeyValueBSONInput(fh=f)
for dict_data in stream:
    print dict_data
f.close()