在 HPC 集群中调用 ncdf4 nc_open 时，R 挂起

Question

I have started using R ( R/3.5.1-foss-2018b) on an HPC server. It seems to work fine except when it has to deal with NetCDF files. I am using the nc_open function from ncdf4 package to read NetCDF file.When I call this function (no matter the size of the NetCDF file), the programme just hangs and I completely loose any control on it. However, with the same version of R on my own laptop, it works fine (so it does not seem to be a R bug). I rather think there is a problem of compatibility with the modules (dependencies) installed on the cluster but I am not able to find out by which one and why the problem occurs. This what I get when I type sessionInfo ():

    R version 3.5.1 (2018-07-02)
    Platform: x86_64-pc-linux-gnu (64-bit)
    Running under: CentOS Linux 7 (Core)

    >Matrix products: default
    BLAS/LAPACK: /mnt/ebinstall/software/OpenBLAS/0.3.1-GCC-7.3.0-2.30/lib/libopenblas_haswellp-r0.3.1.so

    locale:
     [1] LC_CTYPE=en_GB.UTF-8       LC_NUMERIC=C              
     [3] LC_TIME=en_GB.UTF-8        LC_COLLATE=en_GB.UTF-8    
     [5] LC_MONETARY=en_GB.UTF-8    LC_MESSAGES=en_GB.UTF-8   
     [7] LC_PAPER=en_GB.UTF-8       LC_NAME=C                 
     [9] LC_ADDRESS=C               LC_TELEPHONE=C            
    [11] LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C       

    attached base packages:
    [1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_3.5.1

I can get the list of modules currently loaded using module list :

1) easybuild/software

2) sge/8.5.5

3) GCCcore/7.3.0

4) binutils/2.30-GCCcore-7.3.0

5) GCC/7.3.0-2.30

6) zlib/1.2.11-GCCcore-7.3.0

7) numactl/2.0.11-GCCcore-7.3.0

8) XZ/5.2.4-GCCcore-7.3.0

9) libxml2/2.9.8-GCCcore-7.3.0

10) libpciaccess/0.14-GCCcore-7.3.0

11) hwloc/1.11.10-GCCcore-7.3.0

12) OpenMPI/3.1.1-GCC-7.3.0-2.30

13) OpenBLAS/0.3.1-GCC-7.3.0-2.30

14) gompi/2018b

15) FFTW/3.3.8-gompi-2018b

16) ScaLAPACK/2.0.2-gompi-2018b-OpenBLAS-0.3.1

17) foss/2018b

18) bzip2/1.0.6-GCCcore-7.3.0

19) expat/2.2.5-GCCcore-7.3.0

20) libpng/1.6.34-GCCcore-7.3.0

21) freetype/2.9.1-GCCcore-7.3.0 

22) ncurses/6.1-GCCcore-7.3.0

23) util-linux/2.32-GCCcore-7.3.0

24) fontconfig/2.13.0-GCCcore-7.3.0

25) X11/20180604-GCCcore-7.3.0

26) GMP/6.1.2-GCCcore-7.3.0

27) nettle/3.4-foss-2018b

28) libdrm/2.4.92-GCCcore-7.3.0

29) LLVM/6.0.0-GCCcore-7.3.0

30) Mesa/18.1.1-foss-2018b

31) libGLU/9.0.0-foss-2018b

32) pixman/0.34.0-GCCcore-7.3.0

33) libffi/3.2.1-GCCcore-7.3.0

34) gettext/0.19.8.1-GCCcore-7.3.0

35) PCRE/8.41-GCCcore-7.3.0

36) GLib/2.54.3-GCCcore-7.3.0

37) cairo/1.14.12-GCCcore-7.3.0

38) libreadline/7.0-GCCcore-7.3.0

39) Tcl/8.6.8-GCCcore-7.3.0

40) SQLite/3.24.0-GCCcore-7.3.0

41) NASM/2.13.03-GCCcore-7.3.0

42) libjpeg-turbo/2.0.0-GCCcore-7.3.0

43) LibTIFF/4.0.9-GCCcore-7.3.0

44) Java/1.8.0_181(1.8)

45) Tk/8.6.8-GCCcore-7.3.0

46) cURL/7.60.0-GCCcore-7.3.0

47) Python/3.6.6-foss-2018b

48) Szip/2.1.1-GCCcore-7.3.0

49) HDF5/1.10.2-foss-2018b

50) netCDF/4.6.1-foss-2018b

51) GEOS/3.6.2-foss-2018b-Python-3.6.6

52) JasPer/2.0.14-GCCcore-7.3.0

53) PROJ/5.0.0-foss-2018b

54) libgeotiff/1.4.2-foss-2018b

55) GDAL/2.2.3-foss-2018b-Python-3.6.6

56) NLopt/2.4.2-GCCcore-7.3.0

57) libsndfile/1.0.28-GCCcore-7.3.0

58) ICU/61.1-GCCcore-7.3.0

59) UDUNITS/2.2.26-foss-2018b

60) R/3.5.1-foss-2018b

I used to have a similar issue with ncdump but I solved it in loading this module: module load netCDF/4.4.1-intel-2016b .

Furthermore, when I add the following module : module load HDF5/1.10.2-foss-2018b I also get a error message indicating that there is a mismatching between the HDF libraries. This is it:

Warning! ***HDF5 library version mismatched error***
The HDF5 header files used to compile this application do not match
the version used by the HDF5 library to which this application is linked.
Data corruption or segmentation faults may occur if the application continues.
This can happen when an application was compiled by one version of HDF5 but linked with a different version of static or shared HDF5 library.
You should recompile the application or check your shared library related settings such as 'LD_LIBRARY_PATH'. You can, at your own risk, disable this warning by setting the environment variable 'HDF5_DISABLE_VERSION_CHECK' to a value of '1'.
Setting it to 2 or higher will suppress the warning messages totally.
Headers are 1.10.1, library is 1.10.2

That's why I strongly believe that my "hanging" problem is caused by some modules. Does anyone have a clue of what can be a solution?

Answer 1

Try:

export HDF5_DISABLE_VERSION_CHECK=1 

before running the R script.

You can use values higher than 1 to supress the warning totally.