[英]scheduled cores … did not deliver results, all values of the jobs will be affected in parallel::mclapply() in R 4.0.1
I'm using parallel::mclapply()
with R 4.0.1 and getting the following warning: " scheduled cores... did not deliver results, all values of the jobs will be affected ".我正在将
parallel::mclapply()
与 R 4.0.1 一起使用,并收到以下警告:“预定的核心......没有提供结果,所有作业的值都会受到影响”。
Here the result of my investigation: inspecting the function source code, I realized that it happens when the vector dr
is not all TRUE.这是我的调查结果:检查 function 源代码,我意识到当向量
dr
不全为 TRUE 时会发生这种情况。 This means that for some cores the second condition inside the for loop below ( is.raw(a)
) is never executed.这意味着对于某些内核,永远不会执行下面的 for 循环中的第二个条件 (
is.raw(a)
)。 a
is the value returned by readChild()
, that if returned raw data at least once, the condition would be verified at least once. a
是readChild()
返回的值,如果至少返回一次原始数据,则条件将至少验证一次。 So I'm thinking that readChild()
is returning NULL.所以我认为
readChild()
正在返回 NULL。
readChild and readChildren return a raw vector with a "pid" attribute if data were available, an integer vector of length one with the process ID if a child terminated or NULL if the child no longer exists (no children at all for readChildren).
如果数据可用, readChild 和 readChildren 返回一个带有“pid”属性的原始向量,如果孩子终止,则返回一个长度为 1 的 integer 向量,如果孩子终止,则返回 NULL(对于 readChildren 根本没有孩子)。
I ask you to validate or reject my conclusions.我请你验证或拒绝我的结论。 Finally if true what are the possible reasons?
最后,如果属实,可能的原因是什么?
while (!all(fin)) {
s <- selectChildren(ac[!fin], -1)
if (is.null(s)) break # no children -> no hope we get anything (should not happen)
if (is.integer(s))
for (ch in s) {
a <- readChild(ch)
if (is.integer(a)) {
core <- which(cp == a)
fin[core] <- TRUE
} else if (is.raw(a)) {
core <- which(cp == attr(a, "pid"))
job.res[[core]] <- ijr <- unserialize(a)
if (inherits(ijr, "try-error"))
has.errors <- c(has.errors, core)
dr[core] <- TRUE
} else if (is.null(a)) {
# the child no longer exists (should not happen)
core <- which(cp == ch)
fin[core] <- TRUE
}
}
}
This error message can occur when the child process dies/crashes, eg当子进程死亡/崩溃时,可能会出现此错误消息,例如
> y <- parallel::mclapply(1:2, FUN = function(x) if (x == 1) quit("no") else x)
Warning message:
In parallel::mclapply(1:2, FUN = function(x) if (x == 1) quit("no") else x) :
scheduled core 1 did not deliver a result, all values of the job will be affected
> str(y)
List of 2
$ : NULL
$ : int 2
That a child process completely dies is of course not good.一个子进程完全死掉当然不好。 It can happen for several reasons.
这可能有几个原因。 My best guess is that you parallelize something that must not be parallelized.
我最好的猜测是你并行化了一些不能并行化的东西。 Forked processing (=
mclapply()
) is known to be unstable with code that multi-thread, among other things.众所周知,分叉处理(=
mclapply()
)对于多线程代码等不稳定。
For what's it worth, if you use the future framework instead (disclaimer: I'm the author), you'll get an error message that is a bit more informative, eg对于它的价值,如果您改用未来的框架(免责声明:我是作者),您将收到一条信息更丰富的错误消息,例如
> library(future.apply)
> plan(multicore)
> y <- future_lapply(1:2, FUN = function(x) if (x == 1) quit("no") else x)
Error: Failed to retrieve the result of MulticoreFuture (future_lapply-1) from
the forked worker (on localhost; PID 19959). Post-mortem diagnostic: No process
exists with this PID, i.e. the forked localhost worker is no longer alive.
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.