简体   繁体   中英

How can I scrape an embedded tweet? [R]

I am trying to can scrape an embedded tweet on a website. I believe the tweet is loaded via JSON. Ideally, I would be able to simply scrape the embedded tweet's ID. As far as I can tell, this data should be available with the css selector '#twitter-widget-0,' but nothing is returned when I scrape using rvest.

My code is below:

page <- "https://deutsch.rt.com/amerika/86714-rund-woche-nach-russland-auch-china-schickt-militaer-nach-venezuela/"

read_html(page) %>%
  html_nodes('#twitter-widget-0') %>%
    html_text()

Something like this might help

library(dplyr)
library(rvest)

page %>%
  read_html() %>%
  html_nodes("div.rtcode") %>%
  html_text()

#[1] "#Venezuela#China#Russia#Caracas#Chinese army soldiers arrived in 
#Venezuela #Chinese People’s Liberation Army soldiers, as part of a 
#cooperation program, #arrived, after delivering humanitarian supplies, to one 
#of Venezuelan military #facilities. pic.twitter.com/HwZ9Ee67d0— Sukhoi Su-57 
#frazor\U0001f1f7\U0001f1fa\U0001f1ee\U0001f1f3 (@I30mki) 1. April 2019"

Or if you want the unique twitter URL

page %>%
  read_html() %>%
  html_nodes("div.rtcode a") %>%
  html_attr("href") %>%
  grep("status", ., value = TRUE)

#[1] "https://twitter.com/I30mki/status/1112578904835981312?ref_src=twsrc%5Etfw"

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM