[英]How to scrape data generated by javascript using python
I want to scrape the Number of participants of the following news. 我想抓紧以下新闻的参加人数。 The url is http://news.sina.com.cn/c/2013-07-11/175827642839.shtml And I want to get the Number 820. It is generated by javascript. 网址是http://news.sina.com.cn/c/2013-07-11/175827642839.shtml ,我想获取数字820。它是由javascript生成的。 How can I get that number using simple way? 如何使用简单的方法获取该号码?
You could analize javascript code and do the same in python. 您可以分析javascript代码,并在python中执行相同的操作。 Or you can use Selenium in Python. 或者,您可以在Python中使用Selenium 。
edit: 编辑:
Here example from selenium page changed to do what you need. 在这里,硒页面的示例已更改为您所需的内容。
It open browser (firefox), wait 5 second (to load page) and get text 它打开浏览器(firefox),等待5秒钟(加载页面)并获取文本
#!/usr/bin/python
import selenium
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.keys import Keys
import time
browser = webdriver.Firefox() # Get local session of firefox
browser.get("http://news.sina.com.cn/c/2013-07-11/175827642839.shtml ") # Load page
time.sleep(5) # Let the page load
try:
element = browser.find_element_by_xpath("//span[contains(@class,'f_red')]") # get element on page
print element.text # get element text
except NoSuchElementException:
assert 0, "can't find f_red"
browser.close()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.