how to use selenium in a page that doesn`t ever load completely?

Question

I was trying to get some elements from this page: https://www.humblebundle.com/store/shadowgrounds-survivor so I could do something with them. When I inspected the page from Chrome or Firefox I would get every element in the page, or at least I thought so, but when trying to find the element with selenium it just didn't.

I realised the page was loading forever so I need to know if there's any possible way to do something with selenium even if the page isn't fully loaded.

This is my code:

public static void main(String[] args) throws IOException, InterruptedException {
    System.setProperty("webdriver.gecko.driver", "C:\\WebDriver\\bin\\geckodriver.exe");
    DesiredCapabilities capabilities = DesiredCapabilities.firefox();
    capabilities.setCapability("marionette", true);
    WebDriver driver = new FirefoxDriver(capabilities);

    try {
        driver.get("https://www.humblebundle.com/store/shadowgrounds-survivor");
        WebElement listElement1 = driver.findElement(By.tagName(".current-price"));

    } finally {
        driver.close();
    }
}

I get the NoSuchElementException from selenium.

Answer 1

1. There is no need to set capabilities ("marionette", true) the reason is here
2. Selector is wrong By.tagName(".current-price") . Its a class name not tag name so use className selector driver.findElement(By.className(".current-price"))
3. While open the link you shared, it ask for confirm the age as shown below So you need to first handle this

4. Introduce Implicit or Explicit wait in you script to manage timeout in your scripts

So now you code become:

public static void main(String[] args) throws IOException, InterruptedException {
    System.setProperty("webdriver.gecko.driver", "C:\\WebDriver\\bin\\geckodriver.exe");
    WebDriver driver = new FirefoxDriver();
    driver.get("https://www.humblebundle.com/store/shadowgrounds-survivor");
    // handle age confirmation on above suggested page
    WebDriverWait wait = new WebDriverWait(driver,30);
    wait.until(ExpectedConditions.presenceOfElementLocated(By.className(".current-price")));
    driver.get("https://www.humblebundle.com/store/shadowgrounds-survivor");
    WebElement listElement1 = driver.findElement(By.className(".current-price"));
    System.out.println(listElement1.getText());
    driver.close();
}

Answer 2

You can make use of Selenium wait commands where you can wait for time interval for page to load.

WebDriverWait wait = new WebDriverWait(webDriver, timeoutInSeconds);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.id<locator>));