[英]Read large csv file in R
Would you like to tell how to read 3 gb csv file, because when I try to read with read.csv I stumble upon on this error “Error: cannot allocate vector of size 250.0 Mb”. 你想告诉如何阅读3 gb csv文件,因为当我尝试用read.csv阅读时,我偶然发现了这个错误“错误:无法分配大小为250.0 Mb的向量”。 My machine is i5 4 gb ram Win 7 32 bit.
我的机器是i5 4 gb ram Win 7 32 bit。 Moreover, I try to use ff package to open it and again I face error that I should use 64 bit version of R to be able to read csv file as well as I need more ram to read it.
此外,我尝试使用ff包打开它,我再次面临错误,我应该使用64位版本的R能够读取csv文件以及我需要更多ram来阅读它。 Should I take into account to use scan function.
我应该考虑使用扫描功能。 I try to use scan function but I want to use skip = 4 and nlines = 2 in my code and when I try to use it I read only first row of my data.
我尝试使用扫描功能,但我想在我的代码中使用skip = 4和nlines = 2,当我尝试使用它时,我只读取数据的第一行。
R operates completely in memory (RAM), so once it runs out of available space, the read.csv
operation will fail hard. R完全在内存(RAM)中运行,因此一旦可用空间
read.csv
, read.csv
操作就会很难。 To get around this, you can try increasing the amount of available memory: 要解决这个问题,您可以尝试增加可用内存量:
memory.limit(size=8097) # allocate 8097 MB = 8 GB for R to use
You will need the 64 bit version of R to take advantage of this. 您将需要64位版本的R来利用它。 And you may have to increase the memory beyond 8 GB to get your operation to work.
并且您可能必须将内存增加到8 GB以上才能使您的操作正常运行。
如果您尝试使用当前的内存管理方法完成需要过多随机存取内存的分析,那么我建议您调整分析以利用虚拟内存 ,只要您有足够的硬盘空间即可。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.