[英]Using R for web scraping
I need to get the list from a lot of webpages like this: https://fossilplants.info/genus.htm?page=3 I tried to do that using several R packages, such as rvest and XML, but didn't figure out how to make it work.我需要从很多这样的网页中获取列表: https ://fossilplants.info/genus.htm?page =3我尝试使用几个 R 包(例如 rvest 和 XML)来做到这一点,但没有想到出如何使它工作。 Could anyone help me on that?
有人可以帮我吗? Thanks so much.
非常感谢。
We can use rvest
like :我们可以像这样使用
rvest
:
library(rvest)
library(purrr)
url <- 'https://fossilplants.info/genus.htm?page=3'
url %>%
read_html() %>%
html_nodes('h1') %>%
html_text() %>%
gsub('[\r\n\t]', '', .)
# [1] "Genus Abies-pollenites Thierg. in Raatz Abh. Preuss. Geol. Landesanst., Neue Folge, (183): 16. 26 Jan 1938"
# [2] "Genus Abieticedripites Maljavk. Trudy Vsesoyuzn. Neft. Nauchno-Issl. Geol.-Razved. Inst., N. S., (119): 103. 11 Jul 1958"
# [3] "Genus Abietineae-pollenites R. Potonié Palaeontographica, Abt. B, Paläophytol., 91(5-6): 144, 145. Apr 1951"
# [4] "Genus Abietineaepollenites R. Potonié in Delcourt, Sprumont Mém. Soc. Belge Géol., N. Sér. 4°, (5): 51. 1955"
# [5] "Genus Abietipites Wodehouse Bull. Torrey Bot. Club, 60(7): 491. Oct 1933"
# [6] "Genus Abietites Maljavk. Trudy Vsesoyuzn. Neft. Nauchno-Issl. Geol.-Razved. Inst., N. S., (231): 142. 10 Aug 1964"
# [7] "Genus Abietites Hising. Lethaea Svecica 110. 7 Dec 1836"
# [8] "Genus Abietopitys Kräusel Beitr. Geol. Erforsch. Deutsch. Schutzgeb., (20): 32. 11 Aug 1928"
# [9] "Unranked Abietosaccites Erdtman Svensk Bot. Tidskr., 41(1): 110. 26 Mar 1947"
#[10] "Genus Abietoxylon 73. "
If you want to do this for multiple pages, you can change the url like and perform the same function.如果您想对多个页面执行此操作,您可以更改 url 并执行相同的功能。
map(1:3, ~{
url <- sprintf('https://fossilplants.info/genus.htm?page=%d', .x)
url %>%
read_html() %>%
html_nodes('h1') %>%
html_text() %>%
gsub('[\r\n\t]', '', .)
}) %>% flatten_chr()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.