find_elements_by_xpath 仅在某些代码行上返回空值

Question

this is my first question so please let me know if there is anything I should change:)

I am trying to use Selenium Webdriver to obtain a list with all the "Unfollow" buttons in a specific GitHub 'Followers' tab https://github.com/toxtli?page=1&tab=followers , using the following code:

from selenium import webdriver
import time
import sys
from importlib import reload

reload(sys)

driver = webdriver.Firefox()

driver.get("http://github.com/login")
username = driver.find_element_by_id("login_field")
password = driver.find_element_by_id("password")

username.send_keys("my_GH_username")
time.sleep(1)
password.send_keys("my_GH_password")
time.sleep(1)

login_form = driver.find_element_by_xpath("//input[@value='Sign in']")
time.sleep(1)
login_form.click()
time.sleep(1)

prepend = ["toxtli"]
    
for user in prepend:
    for t in range(1, 3):
        string = "https://github.com/{}?tab=followers&page={}".format(user, t)
        driver.get(string)
        time.sleep(1)

        unfollow_button = driver.find_elements_by_xpath('//*[@value="Unfollow"]')
        print(unfollow_button)

        time.sleep(1)

driver.close()

However, no matter what I try the find_elements_by_xpath function will return an empty list [], while I would expect finding several matches like the one in line 523 here: view-source: https://github.com/toxtli?page=1&tab=followers

HTML code for the "Unfollow" button(s) I want to read

Please note that the function seems to work fine if I try with a different line , eg replacing the code below to find the " span " in line 515 :

unfollow_button = driver.find_elements_by_xpath('//*[@class="d-table-cell v-align-middle lh-condensed"]')

Why is this happening, and how can I fix it? Thank you!

Answer 1

from selenium import webdriver
from selenium.webdriver.common.by import By
from time import sleep

driver_path = './geckodriver' #Firefox Executable path
driver = webdriver.Firefox(executable_path=driver_path)

url = "https://github.com/toxtli?page=1&tab=followers" # tesing only for one url

driver.get(url)
sleep(2)

# It will get all unfollow buttons but we need only the ones which are visable so we can filter it
unfollow_btns = driver.find_elements(By.XPATH, "//form//input[@value='Unfollow' and @type='submit']")

for btn in unfollow_btns:
    if btn.is_displayed(): # Checks if unfollow button is displayed
        # Do somthing
        print("We Found Unfollow Btn")

Answer 2

I tried your code (thank you for adding the login piece), using my username/pw and I got all the elements returned in the list. I am not going to print them all here, but I will show you some below that got returned. The only difference is I used Chrome, not Firefox.

[<selenium.webdriver.remote.webelement.WebElement (session="d10763477be39488bf3ed4c4aaae278a", element="b306a8a3-1b31-4ec3-b7b7-e141cdd5497f")>, <selenium.webdriver.remote.webelement.WebElement (session="d10763477be39488bf3ed4c4aaae278a", element="1e5d4b03-104f-4af0-9068-3e30ba19dee0")>, <selenium.webdriver.remote.webelement.WebElement (session="d10763477be39488bf3ed4c4aaae278a", element="567e5d03-6d93-4a01-ac74-1fa7c6e9977f")>, <selenium.webdriver.remote.webelement.WebElement (session="d10763477be39488bf3ed4c4aaae278a", element="028d8212-80cb-4833-aa38-18b9fcb60ab4")>, <selenium.webdriver.remote.webelement.WebElement (session="d10763477be39488bf3ed4c4aaae278a", element="89205c40-9f5f-4d97-89da-621633b14276")>, <selenium.webdriver.remote.webelement.WebElement (session="d10763477be39488bf3ed4c4aaae278a", element="23431ba1-f5ed-41cc-8f1e-e7912e98864a")>, <selenium.webdriver.remote.webelement.WebElement (session="d10763477be39488bf3ed4c4aaae278a", element="6cf602d9-3f28-4593-aac6-83a8cce5edb4")>, <selenium.webdriver.remote.webelement.WebElement (session="d10763477be39488bf3ed4c4aaae278a", element="fccbd345-8090-456a-9604-7af8dc853f9d")>, <selenium.webdriver.remote.webelement.WebElement (session="d10763477be39488bf3ed4c4aaae278a", element="4afec6fe-0193-4b39-b710-c4080380247b")>, <selenium.webdriver.remote.webelement.WebElement (session="d10763477be39488bf3ed4c4aaae278a", element="0e8f6379-ce1f-4369-9303-f24e1a570ba3")>, <selenium.webdriver.remote.webelement.WebElement (session="d10763477be39488bf3ed4c4aaae278a", element="81705790-8358-4dbe-a12c-7b56c3938d7a")>, <selenium.webdriver.remote.webelement.WebElement (session="d10763477be39488bf3ed4c4aaae278a", element="0fe277cb-d52f-48de-9b31-f9a41d4edc78")>, <selenium.webdriver.remote.webelement.WebElement (session="d10763477be39488bf3ed4c4aaae278a", element="906f27b3-a6a7-4833-9647-b978bb956091")>, <selenium.webdriver.remote.webelement.WebElement (session="d10763477be39488bf3ed4c4aaae278a", element="303ce9a1-d143-4f9b-b328-1fbd188435e8")>, <selenium.webdriver.remote.webelement.WebElement...