如何使用Java绑定并行化Selenium Webdriver实例？

Question

I got this for loop that scraps a big list of urls :

//Before

public void testScrapjv() throws Exception {

    List<String> urlList = BIG_LIST_OF_URLS

        for (String url : urlList) {

            driver.get(url);

            //scraping code

}

I'm looking for a way to parallelize this for loop in order to speed up the process of visiting and scraping evey website from the list.

I could split my list in let's say 4 then run manually 4 instances of the program but I don't like the idea of doing things manually. Would someone have an idea on how to do it elegently, the solution can involve shell scripting of course (I have no skill in that domain) but doing it inside the java code would be even better.

I read that I could use selenium-grid maybe but I have no idea what it is and the website doesn't seem to be working ( http://selenium-grid.seleniumhq.org/ ), maybe someone would have some hints.

Answer 1

Split the urlList into no. of instances you want(this has to be done manually, since url's are stored in a single list)

And Use the TestNG framework to run it in parallel.

parallel='tests' This will run your instances in parallel.