如何使用R读取文件夹中的所有hdf文件？

Question

I have thousands of hdf files in a folder. Is there a way to create a loop to read all of the hdf files in that folder and write some specific data to another file?

I read the first file in the folder using the code below:

mydata <- h5read("/path to file/name of the file.he5", "/HDFEOS/GRIDS/Northern Hemisphere/Data Fields/SWE_NorthernDaily")

But I have 1686 more files in the folder, and it is not possible to read one by one. I think I need to write a for loop to read all files in the folder.

I found some codes listing the txt files in a folder and then, read all the files:

nm <- list.files(path="path/to/file")
do.call(rbind, lapply(nm, function(x) read.table(file=x)[, 2]))

I tried to change the code as seen below:

nm <- list.files(path="path/to/file")
do.call(rbind, lapply(nm, function(x) h5read(file=x)[, 2]))

But the error message says:

Error in h5checktypeOrOpenLoc(file, readonly = TRUE, native = native) : Error in h5checktypeOrOpenLoc(). Cannot open file. File 'D:\\path to file\\name of the file.he5' does not exist.

What should I do in that situation?

Answer 1

If you are not bound to a specific technology, you may want to take a look at HDFql . Using HDFql in R, your issue can be solved as follows (for the sake of this example, assume that (1) dataset /HDFEOS/GRIDS/Northern Hemisphere/Data Fields/SWE_NorthernDaily exists in all the HDF5 files stored in the directory, (2) it has one dimension (size 1024), and (3) is of data type integer):

# load HDFql R wrapper (make sure it can be found by the R interpreter)
source("HDFql.R")

# create variable "values" and initialize it
values <- array(dim = c(1024))
for(x in 1:1024)
{
    values[x] <- as.integer(0)
}

# show (i.e. get) files stored in directory "/path/to/hdf5/files" and populate HDFql default cursor with it
hdfql_execute("SHOW FILE /path/to/hdf5/files")

# iterate HDFql default cursor
while(hdfql_cursor_next() == HDFQL_SUCCESS)
{
    file_name <- hdfql_cursor_get_char()

    # select (i.e. read) data from dataset "/HDFEOS/GRIDS/Northern Hemisphere/Data Fields/SWE_NorthernDaily" and populate variable "values" with it
    hdfql_execute(paste("SELECT FROM", file_name, "\"/HDFEOS/GRIDS/Northern Hemisphere/Data Fields/SWE_NorthernDaily\" INTO MEMORY", hdfql_variable_transient_register(values)))

    # display values stored in variable "values"
    for(x in 1:1024)
    {
        print(values[x])
    }
}

Additional examples on how to read datasets using HDFql can be found in the quick start guide and reference manual .