[英]What is the fastest way to load huge .Rdata files (R dataframes) into SQL Server?
I am currently using library(RODBC) via: 我目前正在通过以下方式使用库(RODBC):
dbhandle <- odbcDriverConnect('driver={SQL Server};server=xxxxxx;database=yyyy;trusted_connection=true')
sqlSave(dbhandle, R_table ,tablename = "SQL_table" )
The problem with this one is, it takes about 5 hours to load 10 Million rows into SQL Server. 这个问题是,将1000万行加载到SQL Server中大约需要5个小时。
Also it breaks while loading sometimes. 有时在加载时也会断裂。 So, I break the .Rdata into smaller chunks and sequentially load it into SQL Server.
因此,我将.Rdata分成较小的块,然后依次将其加载到SQL Server中。
Is there any quicker way to achieve this ? 有没有更快的方法来实现这一目标?
I would export my data in CSV format and use BULK INSERT
. 我将以CSV格式导出数据并使用
BULK INSERT
。 If you really want (or have) to use sqlSave, then, the only options that come to my mind are: 如果您真的想(或必须)使用sqlSave,那么我想到的唯一选项是:
fast=TRUE
in sqlSave fast=TRUE
odbcSetAutoCommit(dbhandle, autoCommit = FALSE)
and commit after sqlSave odbcEndTran(dbhandle, commit = TRUE)
odbcSetAutoCommit(dbhandle, autoCommit = FALSE)
并在sqlSave之后提交odbcEndTran(dbhandle, commit = TRUE)
But, in my opinion, the real solution is BULK INSERT. 但是,在我看来,真正的解决方案是批量插入。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.