Selenium Webdriver skips rows in table

Question

I am using HTMLUnitDriver for data extraction. I have a table with rows which is spread on more pages. Like for 50 results, page 1 contains 24 results, page 2 also contains 24 results and page 3 contains 2 results, which all sumed up make 50 results. The problem that I have is that when I try to take all the rows from a page, I might not get all (24 rows), I could get any number less than 24, which I don't want. I always want 24 rows, except for the last page which has less than 24 rows.

The rows in the table have the following form:

<tr id="row28984" class="line_1" onclick="selectRow(28984);Field.activate('qty28984')">
<tr id="row93700" class="line_2" onclick="selectRow(93700);Field.activate('qty93700')">

I used xpath for taking them:

xpath1: //*[@id='resultTable']/tbody/tr
xpath2: /html/body/center/table/tbody/tr[3]/td/form/table[2]/tbody/tr

I also used cssSelector like this:

webDriver.findElements(By.cssSelector(".line_1,.line_2")

but none of what I tried is working. I also tried to wait for the page to completly load:

webDriver.manage().timeouts().pageLoadTimeout(20, TimeUnit.SECONDS)

and tried to wait for the rows to load but nothing changed.

One thing that I think it might work is putting Thread.sleep(1000) but I don't want to do this because it's not a best practice in my case and I want to use something offered by Selenium Webdriver.

Is there a solution to my problem?

Answer 1

try alternative to Thread.sleep - fluentWait : fluentWait function returns your found web element. From the documentation on fluentWait: An implementation of the Wait interface that may have its timeout and polling interval configured on the fly. Each FluentWait instance defines the maximum amount of time to wait for a condition, as well as the frequency with which to check the condition. Furthermore, the user may configure the wait to ignore specific types of exceptions whilst waiting, such as NoSuchElementExceptions when searching for an element on the page. Basic idea of different wait types in selenium you can get here in details .

fluentWait function to use (call):

 public WebElement fluentWait(final By locator) {
        Wait<WebDriver> wait = new FluentWait<WebDriver>(driver)
        .withTimeout(30, TimeUnit.SECONDS)
        // .pollingEvery(5, TimeUnit.SECONDS)
        .pollingEvery(1, TimeUnit.SECONDS)
        // .ignoring(NoSuchElementException.class);
        .ignoring(org.openqa.selenium.NoSuchElementException.class);

        WebElement foo = wait.until(
               new Function<WebDriver, WebElement>() {
            public WebElement apply(WebDriver driver) {
                return driver.findElement(locator);
            }
        }
        );
        return foo;
    }

NOTE: it is up for to adjust polling time interval.

So for you case I would try the following:

String elem1Selector="tr[id*=\"row\"][class=\"line_1\"]";
String elemt2Selector="tr[id*=\"row\"][class=\"line_2\"]";

Approach 1

// next step is to apply fluent wait:

elem1=fluentWait(By.cssSelector(elem1Selector));
elem2=fluentWait(By.cssSelector(elem1Selector)); 

Approach 2 Also we can try to use here explicit wait mechanism which is implemented in this way:

WebDriverWait wait = new WebDriverWait(webDriver, timeoutInSeconds);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.id<locator>));

or

wait.until(ExpectedConditions.elementToBeClickable(By.id<locator>));

projecting to your table it be like:

wait.until(ExpectedConditions.elementToBeClickable(By.cssSelector(elem1Selector)));

Hope this helps you.

Answer 2

Did you try to wait till the rows count captured by selenium is 24 ?

long startTime = System.currentTimeMillis();
    do  {
            int size=driver.findElements(By.xpath("//*[@id='resultTable']/tbody/tr")).size();
            if(size>24)
                break;
        }while(System.currentTimeMillis()-startTime<10000);

you can adjust the time for waiting in while statement.