How to optimize the scroll-down code in java using selenium

Question

I am working a project in MAVEN using Java . I have to get a URL, scroll them down ,and get all the links of other items in this given web page.

Till now, I get the page dynamically using Selenium , and scrolling them down, and fetch the links also. But it takes too much time. Please help me in optimize that.

Example:-, I am working on a page , whose link is here .

My Questions :-

1. Scrolling web page using selenium is very slow. How can I optimize this? (Suggest any other method
   to do the same or help me to optimize this one)

Thanks in advance. Looking for your kind response.

Code to dynamically get and scroll the page :-

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import java.io.IOException;
import com.google.common.collect.*;
import java.io.File;
import java.util.ArrayList;
import java.util.Date;
import org.apache.commons.io.FileUtils;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxProfile;

/**
 *
 * @author jhamb
 */
public class Scroll_down {

    private static FirefoxProfile createFirefoxProfile() {
        File profileDir = new File("/tmp/firefox-profile-dir");
        if (profileDir.exists()) {
            return new FirefoxProfile(profileDir);
        }
        FirefoxProfile firefoxProfile = new FirefoxProfile();
        File dir = firefoxProfile.layoutOnDisk();
        try {
            profileDir.mkdirs();
            FileUtils.copyDirectory(dir, profileDir);
        } catch (IOException e) {
            e.printStackTrace();
        }
        return firefoxProfile;
    }



    public static void main(String[] args)  throws InterruptedException{
        String url1 = "http://www.jabong.com/men/shoes/men-sports-shoes/?source=home-leftnav";
        System.out.println("Fetching %s..." + url1);
        WebDriver driver = new FirefoxDriver(createFirefoxProfile());


        driver.get(url1);  

        JavascriptExecutor jse = (JavascriptExecutor)driver;
        jse.executeScript("window.scrollBy(0,250)", "");
        for (int second = 0;; second++) {
            if (second >= 60) {
                break;
            }
            jse.executeScript("window.scrollBy(0,200)", "");
            Thread.sleep(1000);
        }
            String hml = driver.getPageSource();
        driver.close();


        Document document = Jsoup.parse(hml);

            Elements links = document.select("div");

        for (Element link : links) {
            System.out.println(link.attr("data-url"));
        }
    }
}

Answer 1

Well Selenium scrolling is based on Javascript. I dont know your goal with selenium though, you have no assertion to compare anything in your code ? When you are so sure that your data fetching so fast then don't use any sleep methode. Sleep methods makes selenium slower, but yeah it is waiting until the element is properly loaded ..... It's up to you, what to test though

Answer 2

How about page down?

ele.sendKeys(Keys.PAGE_DOWN);   //WebElement ele = <Any existing element>

Repeat this till you find that particular item.