[英]find element by xpath not defined
When I run当我跑
# change the value inside the range to save the number of reviews we're going to grab
for i in range(0, pages_to_scrape):
# give the DOM time to load
time.sleep(2)
# Click the "expand review" link to reveal the entire review.
find_element_by_xpath(".//div[contains(@data-test-target, 'expand-review')]").click()
# Now we'll ask Selenium to look for elements in the page and save them to a variable. First lets define a container that will hold all the reviews on the page. In a moment we'll parse these and save them:
container = driver.findElement(By.xpath("//div[@data-reviewid]"))
I get the error我得到错误
NameError Traceback (most recent call last)
/var/folders/6c/jpl964752rv_72zjclrp_8ym0000gn/T/ipykernel_18848/426937913.py in <module>
6
7 # Click the "expand review" link to reveal the entire review.
----> 8 find_element_by_xpath(".//div[contains(@data-test-target, 'expand-review')]").click()
9
10 # Now we'll ask Selenium to look for elements in the page and save them to a variable. First lets define a container that will hold all the reviews on the page. In a moment we'll parse these and save them:
NameError: name 'find_element_by_xpath' is not defined
I used print(dir(driver))
to see what command i'm supposed to use, and the one I am using is the correct one.我使用
print(dir(driver))
来查看我应该使用什么命令,而我使用的是正确的命令。 I was originally using driver.findElement(By.xpath(".//div[contains(@data-test-target, 'expand-review')]").click()
, which worked, but when combined with other codes gave the same error above, except it said "By" was not defined. Then I tried a user suggestion driver.find_element(By.XPATH, 'your xpath')
, which also generated the same error.我最初使用的是
driver.findElement(By.xpath(".//div[contains(@data-test-target, 'expand-review')]").click()
,它有效,但与其他代码结合使用上面给出了同样的错误,除了它说“By”没有定义。然后我尝试了一个用户建议driver.find_element(By.XPATH, 'your xpath')
,这也产生了同样的错误。
Full code:完整代码:
import selenium
import csv #This package lets us save data to a csv file
from selenium import webdriver #The Selenium package we'll need
import time #This package lets us pause execution for a bit
path_to_file = "/Users/user/Desktop/HotelReviews.csv"
pages_to_scrape = 3
url = "https://www.tripadvisor.com/Hotel_Review-g60982-d209422-Reviews-Hilton_Waikiki_Beach-Honolulu_Oahu_Hawaii.html"
# import the webdriver
driver = webdriver.Safari()
driver.get(url)
# open the file to save the review
csvFile = open(path_to_file, 'a', encoding="utf-8")
csvWriter = csv.writer(csvFile)
# change the value inside the range to save the number of reviews we're going to grab
from selenium.webdriver.common.by import By
for i in range(0, pages_to_scrape):
# give the DOM time to load
time.sleep(2)
# Click the "expand review" link to reveal the entire review.
driver.find_elements(by=By.XPATH, ".//div[contains(@data-test-target, 'expand-review')]").click()
# When all pages have been processed, quit the driver
driver.quit()
from selenium.webdriver.common.by import By
for i in range(0, pages_to_scrape):
# give the DOM time to load
time.sleep(2)
# Click the "expand review" link to reveal the entire review.
driver = webdriver.Safari() #url was defined in a different portion of the code
driver.get(url)
driver.find_element_by_xpath(".//div[contains(@data-test-target, 'expand-review')]").click()
which generates这会产生
File "/var/folders/6c/jpl964752rv_72zjclrp_8ym0000gn/T/ipykernel_24069/1057492970.py", line 15
driver.find_elements(by=By.XPATH,".//div[contains(@data-test-target, 'expand-review')]").click()
^
SyntaxError: positional argument follows keyword argument
Following the format in the error code, I ran按照错误代码中的格式,我跑了
driver.find_elements(by=By.XPATH, ".//div[contains(@data-test-target, 'expand-review')]").click()
Which then gave another error:然后又给出了另一个错误:
File "/var/folders/6c/jpl964752rv_72zjclrp_8ym0000gn/T/ipykernel_24069/4123117837.py", line 11
driver.find_elements(by=By.XPATH, ".//div[contains(@data-test-target, 'expand-review')]").click()
^
SyntaxError: positional argument follows keyword argument
You are calling find_element_by_xpath
directly.您直接调用
find_element_by_xpath
。
All the selenium methods need to be called from the driver object after you instantiate it, declaring the website you are scraping.实例化后,需要从驱动程序 object调用所有 selenium 方法,声明您正在抓取的网站。
So the steps are:所以步骤是:
get()
to go to the website you want to scrapeget()
到 go 到你要抓取的网站Example:例子:
driver = webdriver.Chrome("path/to/driver")
driver.get("yourwebsite.com")
driver.find_element_by_xpath(".//div[contains(@data-test-target, 'expand-review')]").click()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.