简体   繁体   English

Python:最大数组大小?

[英]Python: Max size of arrays?

I have a script that simply reads the files and add their contents to an array. 我有一个脚本,可以简单地读取文件并将其内容添加到数组中。 each file have a size of 7M to 15M. 每个文件的大小为7M至15M。 and the number of files are 100. in total the size of files are 800M. 文件数量为100。文件大小总计为800M。

full_str = []
for indexfile in files:
    with open(indexfile) as f:
        iindex = f.read()
    full_str += [str(iindex)]

I know from here that the max number of elements should be less than 500M, so it is not a limit here, However I get out of me problem when I run the code. 我从这里知道最大元素数应小于500M,因此这里没有限制,但是我在运行代码时遇到了麻烦。 can any one please tell me if there is another limit that kills the process? 有谁能告诉我是否还有另一个限制可以终止该过程? What should I do? 我该怎么办?

That's surprising, I found that the process get killed because the content of the files contain non ASKI characters, and it results in this error. 令人惊讶的是,我发现该进程被终止了,因为文件的内容包含非ASKI字符,并导致了此错误。 So I simply encoded every thing in 'UTF-8' and now it works. 因此,我只是将所有内容都编码为“ UTF-8”,现在就可以使用了。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM