[英]Writing to database in parallel in R
I try to write a table which is a processed subset of a global data variable, in a normal for loop this piece of code works fine but when I try to do it in parallel it raises an error.我尝试编写一个表,该表是全局数据变量的已处理子集,在正常的 for 循环中,这段代码工作正常,但是当我尝试并行执行时,它会引发错误。
Here is my piece of code;这是我的一段代码;
library(doParallel)
library(foreach)
library(odbc)
library(data.table)
nc <- detectCores() - 1
cs <- makeCluster(nc)
registerDoParallel(cs)
con <- dbConnect(odbc(),driver = 'SQL Server',server = 'localserver',database = 'mydb', encoding = 'utf-8',timeout = 20)
range_to <- 1e6
set.seed(1)
random_df <- data.table(a = rnorm(n = range_to,mean = 2,sd = 1),
b = runif(n = range_to,min = 1,max = 300))
foreach(i=1:1000,.packages = c('odbc','data.table')) %dopar% {
subk <- random_df[i,]
subk <- subk**2
odbc::dbWriteTable(conn = con,name = 'parallel_test',value = subk,row.names = FALSE,append = TRUE)
}
This code raises this error;此代码引发此错误;
Error in {: task 1 failed - "unable to find an inherited method for function 'dbWriteTable' for signature '"Microsoft SQL Server", "character", "data.table"'"
Like I said before in a normal for loop it works fine.就像我之前在普通 for 循环中所说的那样,它工作正常。
Thanks in advance.提前致谢。
I solved that issue by changing only creating connection object method by;我通过仅更改创建连接 object 方法解决了该问题;
parallel::clusterEvalQ(cs, {library(odbc);con <- dbConnect(odbc(),driver = 'SQL Server',server = 'localserver',database = 'mydb', encoding = 'utf-8',timeout = 20)})
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.