使用 rvest 刮 ASX

Question

I'm trying to scrape data from the ASX (Australian Stock Exchange) site. For example, on BHP on ASX , at the bottom of the page is a collection of fundamentals data. The selector for the values, eg eps, is:

#company_key_statistics > div > div.panel-body.row > div:nth-child(3) > table > tbody > tr:nth-child(8) > td

I tried

library(rvest)
ASX_bhp <-read_html("https://www2.asx.com.au/markets/company/bhp")
ASX_data <- ASX_bhp |> html_elements("td") |> html_text()

or instead of "td", I have tried "tr", "#company_key_statistics", or the whole selector string. However, all return an empty character. I also tried html_nodes instead of html_elements .

How should I extract fundamental data from this site?

Answer 1

All that data is fetched and presented through JavaScript, thus it's not available for rvest (at least not through that URL). But you can user their API:

library(jsonlite)
bhp <- fromJSON("https://asx.api.markitdigital.com/asx-research/1.0/companies/bhp/key-statistics")
bhp$data$earningsPerShare
#> [1] 5.95708

^{Created on 2022-09-19 with reprex v2.0.2}