如何从网站上抓取信息

Question

I am trying to scrape data from the PHE website ( https://coronavirus.data.gov.uk/details/deaths ). I am after the number of deaths within 28 days of the positive test by date reported by nation (second interactive). I tried to use selector gadget to pull the data and put it into a table format but it returns zero. I have done this in the past and it worked fine so am not sure why it doesn't work this time. Suspect it might be because it is a kind of interactive dashboard. Any help will be appreciated.

library(rvest)

url <- "https://coronavirus.data.gov.uk/details/deaths"
webpage <‐ read_html(url)

data <- webpage %>%
  html_nodes(".dgxcKs , .govuk-table__cell--date , .govuk-table__cell--numeric , .cQSaWH") %>%
  html_table()

print(data)

Answer 1

All the data on that page is available in the json format. You need to find the relevant json from the network tab of the browser.

data <- jsonlite::fromJSON('https://coronavirus.data.gov.uk/api/v1/data?filters=areaType=overview&structure=%7B%22date%22:%22date%22,%22areaName%22:%22areaName%22,%22newDeaths28DaysByPublishDate%22:%22newDeaths28DaysByPublishDate%22%7D')
head(data$data)

#        date       areaName newDeaths28DaysByPublishDate
#1 2021-03-24 United Kingdom                           98
#2 2021-03-23 United Kingdom                          112
#3 2021-03-22 United Kingdom                           17
#4 2021-03-21 United Kingdom                           33
#5 2021-03-20 United Kingdom                           96
#6 2021-03-19 United Kingdom                          101