将多个脱机html文件读取到R中的列表

Question

I have rawdata as 20 offline html files stored in following format

../rawdata/1999_table.html
../rawdata/2000_table.html
../rawdata/2001_table.html
../rawdata/2002_table.html
.
.
../rawdata/2017_table.html

These files contain tables that I am extracting and reshaping to a particular format.

I want to read these files at once to a list and process them one by one through a function that I have written.

What I tried: I put the names of these files into an Excel file called filestoread.xlsx and used a for loop to load these files using the names mentioned in the sheet. But it doesn't seem to work

filestoread <- fread("../rawdata/filestoread.csv")

x <- list()
for (i in nrow(filestoread)) {
  x[[i]] <- read_html(paste0("../rawdata/", filestoread[i]))
}

How can this be done?

Also, after reading the HTML files I want to extract the tables from them and reshape them using a function I wrote after converting it to a data table.

My final objective is to rbind all the tables and have a single data table with year wise entries of the tables in the html file.

Answer 1

First save path of your data on one of the following ways.

Either, hardcoded

filestoread <- paste0("../rawdata/", 1999:2017, "_table.html")

or reading all html files in the directory

filestoread <- list.files(path = "../rawdata/", pattern="\\.html$")

Then use lapply()

library(rvest)
lapply(filestoread, function(x) try(read_html(x)))

Note: try() runs the code even when there is a file missing (throwing error).

The second part of your question is a little broad, depends on the content of your files, and there are already some answers, you could consider eg this answer . In principle you use a combination of ?html_nodes and ?html_table .