[英]while loop issues in PYthon + Selenium
你能告訴我為什么我的while循環不起作用嗎? 我沒有收到錯誤消息,它只運行一次。
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import pandas as pd
import time
PATH = "/Users/csongordoma/Documents/chromedriver"
driver = webdriver.Chrome(PATH)
current_page = 1
driver.get('https://ingatlan.com/lista/elado+lakas+budapest?page=' + str(current_page))
data = {}
df = pd.DataFrame(columns=['Price', 'Address', 'Size', 'Rooms', 'URL', 'Labels'])
listings = driver.find_elements_by_css_selector('div.listing__card')
while current_page < 10:
for listing in listings:
data['Price'] = listing.find_elements_by_css_selector('div.price')[0].text
data['Address'] = listing.find_elements_by_css_selector('div.listing__address')[0].text
data['Size'] = listing.find_elements_by_css_selector('div.listing__parameters')[0].text
data['Labels'] = listing.find_elements_by_css_selector('div.listing__labels')[0].text
data['URL'] = listing.find_elements_by_css_selector('a.listing__link.js-listing-active-area')[0].get_attribute('href')
df = df.append(data, ignore_index=True)
current_page += 1
print(len(listings))
print(df)
# driver.find_element_by_xpath("//a[. = 'Következő oldal']").click()
driver.quit()
output 是一個很好的數據框,有 20 個項目,相當於一頁。 在我試圖抓取的網站上。 將限制設置為 10 個周期,以免任何人超載,但理想情況下,我想瀏覽所有頁面。
只需將代碼安排在您的 while 循環中,並將當前頁面縮進到外部循環中。 我添加了一個嘗試,除非出現任何錯誤,並且 webdriver 在 driver.get 之后等待獲取元素的一致性。
current_page = 1
data = {}
df = pd.DataFrame(columns=['Price', 'Address', 'Size', 'Rooms', 'URL', 'Labels'])
while current_page < 10:
driver.get('https://ingatlan.com/lista/elado+lakas+budapest?page=' + str(current_page))
try:
listings=WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "div.listing__card")))
for listing in listings:
data['Price'] = listing.find_elements_by_css_selector('div.price')[0].text
data['Address'] = listing.find_elements_by_css_selector('div.listing__address')[0].text
data['Size'] = listing.find_elements_by_css_selector('div.listing__parameters')[0].text
data['Labels'] = listing.find_elements_by_css_selector('div.listing__labels')[0].text
data['URL'] = listing.find_elements_by_css_selector('a.listing__link.js-listing-active-area')[0].get_attribute('href')
df = df.append(data, ignore_index=True)
except:
print('Error')
current_page += 1
print(len(listings))
print(df)
進口
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.