硒嵌套循环到下一页

Question

I have built a script using Selenium that loops through a page, prints the data and goes to the next page and does the same.

Now I am trying to save the data to a CSV file and thus need to create a nested loop - Currently I am repeating the loop multiple times (as below).

How do I create a nested loop then save to the CSV file?

Also will the script fail if it gets the last page and their isn't a next button there?

Thanks - This is the code I am using.

from selenium import webdriver
import time

browser = webdriver.Firefox(executable_path="/Users/path/geckodriver")

browser.get('https://www.tripadvisor.co.uk/Restaurants-g186338-zfn29367-London_England.html#EATERY_OVERVIEW_BOX')


meci = browser.find_elements_by_class_name('property_title')

for items in meci:
    title = items.text
    href = items.get_attribute('href')
    print(title)
    print(href)

time.sleep(3)
browser.find_element_by_css_selector('.next').click()
time.sleep(3)
meci = browser.find_elements_by_class_name('property_title')

for items in meci:
    title = items.text
    href = items.get_attribute('href')
    print(title)
    print(href)

time.sleep(3)
browser.find_element_by_css_selector('.next').click()
time.sleep(3)
meci = browser.find_elements_by_class_name('property_title')

for items in meci:
    title = items.text
    href = items.get_attribute('href')
    print(title)
    print(href)

browser.quit()

Answer 1

I have used try-except so the program will exit when there isn't a next button. Instead of printing, you can write the result to a CSV file.

while True:
    try:
        meci = browser.find_elements_by_class_name('property_title')

        for items in meci:
            title = items.text
            href = items.get_attribute('href')
            print(title)
            print(href)

        time.sleep(3)
        browser.find_element_by_css_selector('.next').click()
        time.sleep(3)
    except:
        break


browser.quit()