How to capture all href link and loop over in python using selenium

Question

I have link I am trying to click on and the value changes each year. I need the /results at the end otherwise other links are selected.

WebDriverWait(driver,15).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR,"span > strong > a[href^='/basketball/sc/']")))

Answer 1

If you want to catch all elements besides this with 2019/2020, try :

elements_list = WebDriverWait(driver,15).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR,"span > strong > a[href^='/basketball/europe/euroleague-'][href*='results']")))

and loop over them:

for element in elements_list:
    element.click()

Or to catch all elements in one list:

elements_list = WebDriverWait(driver,15).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR,"span > strong > a[href^='/basketball/europe/euroleague][href*='results']")))

Answer 2

To get the link text and link href value induce WebDriverWait () and visibility_of_all_elements_located () and following CSS selector .

Get all the text in a list.

allelements=WebDriverWait(driver,20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR,"div.main-menu2.main-menu-gray >ul.main-filter a[href^='/basketball/europe/euroleague']")))

#Get the all year text of link in a list.
alltext=[ele.text for ele in allelements]
print(alltext)

Output :

['2019/2020', '2018/2019', '2017/2018', '2016/2017', '2015/2016', '2014/2015', '2013/2014', '2012/2013', '2011/2012', '2010/2011', '2009/2010', '2008/2009', '2007/2008', '2006/2007', '2005/2006', '2004/2005', '2003/2004', '2002/2003', '2001/2002', '2000/2001', '1999/2000', '1998/1999']

---

Get all the href in a list

#Get the all year href of link in a list.
allyearslink=[ele.get_attribute('href') for ele in allelements]
print(allyearslink)

Output :

['https://www.oddsportal.com/basketball/europe/euroleague/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-2018-2019/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-2017-2018/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-2016-2017/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-2015-2016/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-2014-2015/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-2013-2014/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-2012-2013/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-2011-2012/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-2010-2011/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-2009-2010/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-2008-2009/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-2007-2008/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-2006-2007/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-2005-2006/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-2004-2005/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-2003-2004/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-2002-2003/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-2001-2002/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-2000-2001/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-1999-2000/results/', 'https://www.oddsportal.com/basketball/europe/euroleague-1998-1999/results/']

---

Traverse each page

#To traverse each page iterate here and perform your operation.

for link in allyearslink:
    driver.get(link)
    #perform some operations lets say get the page title.
    print("Page Title: " +driver.title)

Output :

Page Title: Euroleague Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 2018/2019 Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 2017/2018 Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 2016/2017 Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 2015/2016 Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 2014/2015 Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 2013/2014 Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 2012/2013 Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 2011/2012 Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 2010/2011 Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 2009/2010 Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 2008/2009 Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 2007/2008 Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 2006/2007 Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 2005/2006 Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 2004/2005 Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 2003/2004 Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 2002/2003 Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 2001/2002 Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 2000/2001 Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 1999/2000 Results & Historical Odds, Basketball Europe Archive
Page Title: Euroleague 1998/1999 Results & Historical Odds, Basketball Europe Archive

Let me know if you need any further info.Thanks.