如何修復 R 中 ncdf4 並行處理中的 C function R_nc4_get_vara_double 返回錯誤

Question

我想通過 OPENDAP 從遠程存儲下載 nc 數據。 我將並行后端與 foreach - dopar 循環一起使用，如下所示：

# INPUTS
inputs=commandArgs(trailingOnly = T)
interimpath=as.character(inputs[1])
gcm=as.character(inputs[2])
period=as.character(inputs[3])
var=as.character(inputs[4])
source='MACAV2'

cat('\n\n EXTRACTING DATA FOR',var, gcm, period, '\n\n')

# CHANGING LIBRARY PATHS
.libPaths("/storage/home/htn5098/local_lib/R40") # local library for packages
setwd('/storage/work/h/htn5098/DataAnalysis')
source('./src/Rcodes/CWD_function_package.R') # Calling the function Rscript

# CALLING PACKAGES
library(foreach)
library(doParallel)
library(parallel)
library(filematrix)

# REGISTERING CORES FOR PARALLEL PROCESSING
no_cores <- detectCores() 
cl <- makeCluster(no_cores)
registerDoParallel(cl)
invisible(clusterEvalQ(cl,.libPaths("/storage/home/htn5098/local_lib/R40"))) # Really have to import library paths into the workers
invisible(clusterEvalQ(cl, c(library(ncdf4))))

# EXTRACTING DATA FROM THE .NC FILES TO MATRIX FORM
url <- readLines('./data/external/MACAV2_OPENDAP_allvar_allgcm_allperiod.txt')
links <- grep(x = url,pattern = paste0('.*',var,'.*',gcm,'_.*',period), value = T) 

start=c(659,93,1) # lon, lat, time
count=c(527,307,-1)

spfile <- read.csv('./data/external/SERC_MACAV2_Elev.csv',header = T)
grids <- sort(unique(spfile$Grid))

clusterExport(cl,list('ncarray2matrix','start','count','grids')) #exporting data into clusters for parallel processing

cat('\nChecking when downloading all grids\n')

# k <- foreach(x = links,.packages = c('ncdf4')) %dopar% {
    # nc <- nc_open(x)
    # nc.var=ncvar_get(nc,varid=names(nc$var),start=start,count=count)
    # return(nc.var)
    # nc_close(nc)
# }

k <- foreach(x = links,.packages = c('ncdf4'),.errorhandling = 'pass') %dopar% {
    nc <- nc_open(x)
    print(nc)
    nc.var=ncvar_get(nc,varid=names(nc$var),start=c(659,93,1),count=c(527,307,-1))
    nc_close(nc)
    return(dim(nc.var))
    Sys.sleep(10)
}

# k <- parSapply(cl,links,function(x) {
    # nc <- nc_open(x)
    # nc.var=ncvar_get(nc,varid=names(nc$var),start=start,count=count)
    # nc_close(nc)
    # return(nc.var)
# })

print(k)

但是，我不斷收到此錯誤：

<simpleError in ncvar_get_inner(ncid2use, varid2use, nc$var[[li]]$missval, addOffset,     scaleFact, start = start, count = count, verbose = verbose,     signedbyte = signedbyte, collapse_degen = collapse_degen): C function R_nc4_get_vara_double returned error>

這個問題的原因可能是什么？ 您能否為此推薦一個省時的解決方案（我必須對大約 20 個文件重復此操作）？

謝謝你。

Answer 1

我的代碼中有同樣的錯誤。 問題不在於代碼本身。 這是我想閱讀的文件之一。 它有問題，所以 R 無法打開它。 我識別了該文件並再次下載它，並且相同的代碼運行良好。