[英]I am getting error in scraping content through selenium python
我正在通過 selenium 在https://www.indeed.ae/jobs-in-dubai上抓取工作結果的標題。 我認為.text 不起作用。 我正在通過 selenium 其中 go 將代碼運行到主網站,輸入選擇性關鍵字,然后從結果中刮掉所有標題。 但我收到錯誤,我該如何解決這個錯誤
這是我的代碼
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
Path = "C:\Program Files (x86)\chromedriver.exe"
driver = webdriver.Chrome(Path)
driver.get("https://indeed.ae/")
print(driver.title)
search = driver.find_element_by_name("l")
search.send_keys("Dubai")
search.send_keys(Keys.RETURN)
try:
td = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.ID, "resultsCol"))
)
divs = td.find_elements_by_tag_name("div")
for div in divs:
header = div.find_element_by_class_name("title")
print(header)
finally:
driver.quit()
driver.quit()
我收到以下錯誤
Job Search | Indeed
Traceback (most recent call last):
File "C:/Users/hp/Desktop/python projects/selenium-pycharm/selenium-bot.py", line 24, in <module>
header = div.find_element_by_class_name("title")
File "C:\Users\hp\Desktop\python projects\selenium-pycharm\venv\lib\site-packages\selenium\webdriver\remote\webelement.py", line 398, in find_element_by_class_name
return self.find_element(by=By.CLASS_NAME, value=name)
File "C:\Users\hp\Desktop\python projects\selenium-pycharm\venv\lib\site-packages\selenium\webdriver\remote\webelement.py", line 659, in find_element
{"using": by, "value": value})['value']
File "C:\Users\hp\Desktop\python projects\selenium-pycharm\venv\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute
return self._parent.execute(command, params)
File "C:\Users\hp\Desktop\python projects\selenium-pycharm\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\hp\Desktop\python projects\selenium-pycharm\venv\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".title"}
(Session info: chrome=83.0.4103.116)
Process finished with exit code 1
提前致謝
您找不到標題,因為您從 resultsCol 中獲取了所有 div。 這意味着有些 div 有標題,有些沒有。
嘗試這個:
try:
td = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.ID, "resultsCol"))
)
divs = td.find_elements_by_tag_name("div")
#print(divs)
for div in divs:
try:
header = div.find_element_by_class_name("title")
print(header.text)
except:
continue
finally:
driver.quit()
driver.quit()
標題為 output:
Receptionist
Administrative Assistant/ Document Controller
RECEPTIONIST
ADMIN OFFICER IN UAE
Data Entry Assistant (Fresh Graduate)
Receptionist
Replenishment Associate - Light Household - Hypermarket
DOCUMENT CONTROLLER
School Administrative Assistant - Dubai
ACCOUNTANT
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.