Python Selenium driver.find_element().text returns empty string, but text is visible in the driver.page_source

Question

I'm trying to scrape some titles of the videos and to do so I'm using Selenium, but I've encountered a problem. driver.find_element().text returns empty string, but title is for sure located in given XPATH. Here is the fragment of the page source returned by driver.page_source:

<div class="title"><a href="/f/4n3x7e31hpwxm8"target="_blank">Big.Sky.S03E01.ITA.WEBDL.1080p</a></div>

To find the title I am trying to use:

hoverable = driver.find_element(By.XPATH, '//*[@id="videojs"]/div[1]')
ActionChains(driver).move_to_element(hoverable).perform()
wait = WebDriverWait(driver, 20)
title_from_url = wait.until(EC.visibility_of_element_located((By.XPATH, '/html/body/div[1]/a'))).text
title_from_url = driver.find_element(
    By.XPATH, '/html/body/div[1]/a'
    ).text.casefold()

From what I've read it could be caused by the fact that the page might not be fully loaded (I wasn't using any wait condition here). After that I've tried to add a wait condition and even time.sleep(), but it didn't change anything. <mini question: how would proper wait staitment look like here?>

Edit: I think the problem is caused, because title is showing up only when mouse is in the player area. I think some mouse movement will be needed here, but I have tried to move mouse into the player area and for some time it is working but after a while there is a moment when title will disappear too fast. Is there a way to use find_element() while also moving mouse?

Any help will be appreciated. Best regards, Ed.

Example site: https://mixdrop.to/e/4n3x7e31hpwxm8 .

Answer 1

You have to wait for element to be completely loaded before extracting it text content. WebDriverWait expected_conditions explicit waits should be used for that.
This should wait in case the element is visible on the page and the locator is correct:

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

wait = WebDriverWait(driver, 20)

title_from_url = wait.until(EC.visibility_of_element_located((By.XPATH, '//div[contains(@class, "title")]/a'))).text

UPD
In case the element content is dynamically changes and we need to hover over that element to make the desired text to appear there, we can simulate the mouse hover action with the help of ActionChains module.
The tool-tip etc. will not disappear until you perform some click etc. on that page. So, just performing a background action of driver.find_element() will not affect that text.
UPD2
The title element is not visible. It becomes visible only by hovering over the player.
Here I'm performing such hovering and then getting the title text:

from selenium import webdriver
from selenium.webdriver import ActionChains
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = Options()
options.add_argument("start-maximized")

webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(options=options, service=webdriver_service)
wait = WebDriverWait(driver, 5)
actions = ActionChains(driver)

url = "https://mixdrop.to/e/4n3x7e31hpwxm8"
driver.get(url)

player = wait.until(EC.presence_of_element_located((By.CLASS_NAME, 'player')))
actions.move_to_element(player).perform()
title = wait.until(EC.presence_of_element_located((By.XPATH, '//div[contains(@class, "title")]/a')))
print(title.text)

The output is:

Big.Sky.S03E01.ITA.WEBDL.1080p

Answer 2

If you suspect its because of sync issue. You can use selenium waits.Let it be implicit of explicit.

Implicit: objdriver.implicitely_wait(float)

Explicit:

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

objwait=WebDriverWait(driver,float,poll_frequency=float,ignored_exception=float)

objelement=objwait.until(EC.visibility_of_element_located((By.XPATH,"Your XPATH")))