Python Selenium 'WebDriver' object has no attribute error

Question

I'm trying to scrape some javascript-generated content from a Chinese-language website. I'm using Selenium (and Python) since I can't scrape the javascript content directly.

# -*- coding: utf-8 -*-
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.selenium import selenium 

import time
import urllib2
import httplib
import urllib
import re
import base64

browser = webdriver.Firefox() # Get local session of firefox
browser.get("http://www...") # Load page (redacted here, but any works)
browser.get_body_text() #Attempt to scrape body text

And I get the following error:

'WebDriver' object has no attribute 'get_body_text'

In fact, I can't seem to call any commands in the selenium.selenium class. No doubt I'm overlooking something very obvious. Thanks in advance.

Answer 1

1. You only need from selenium import webdriver .
2. Execute html= browser.find_element_by_xpath(".//html") to get the html element on the page, the largest element. (You can do this any number of ways, and select any number of elements. )
3. Execute html.text to return the text of the page.

.text is a method of an element object. Step 2 is the assignment of the element to the name html .