从javascript网页提取文本

Question

I want to use R to extract some text from a website. I am not able to access text using Rvest. The area I am interested in is the section 'Principal Investment Strategies' If I can extract that section I can use Grep to further analyze the text. But obtaining the section in extractable format is proving to be a challenge.

Link to the site is as follows: http://quote.morningstar.com/etf-filing/Summary-Prospectus/2017/8/28/t.aspx?t=AGG&ft=497K&d=c6995d020ec0f1b3592873780a199bd1

Answer 1

Using rvest to extract the complete text of that part (iframe) and perhaps you could use regex or tokenizer to extract the part you want from the text:

link <- 'http://quote.morningstar.com/etf-filing/Summary-Prospectus/2017/8/28/t.aspx?t=AGG&ft=497K&d=c6995d020ec0f1b3592873780a199bd1'
library(rvest)
library(magrittr)
link  %>%
  read_html() %>%
  html_nodes("iframe") %>%
  extract(4) %>%
  html_attr("src") %>% 
  read_html() %>%
  html_text() 

Answer 2

Taking it on faith that you've done what you said (it's hard to be sure with no code sample).

Grabbing that text with precise targeting, starting with the original URL, finding that iframe then just the <div> with that text.

library(rvest)

read_html("http://quote.morningstar.com/etf-filing/Summary-Prospectus/2017/8/28/t.aspx?t=AGG&ft=497K&d=c6995d020ec0f1b3592873780a199bd1") %>% 
  html_node("iframe.sec_frame") %>% 
  html_attr("src") %>% 
  read_html() -> pg

html_node(pg, xpath=".//div[contains(., 'Principal Investment Strategies
')]") %>% 
  html_text()