I'm using selenium webdriver to automate web page usage. Headless browser not allowed.
Selenium seems rather slow at finding multiple elements on a single page that is fully loaded.
Does anyone have any tips for how to speed things up? I am generally searching for objects via xpath.
I have searched google and read similar SO posts . I am looking for new ideas
in this case i like to create a org.w3c.dom Document using the page source and then parse it using the javax.xml libary:
public static Document getWebpageDocument_fromSource(String source) throws InterruptedException, IOException {
try {
HtmlCleaner cleaner = new HtmlCleaner();
CleanerProperties props = cleaner.getProperties();
props.setAllowHtmlInsideAttributes(true);
props.setAllowMultiWordAttributes(true);
props.setRecognizeUnicodeChars(true);
props.setOmitComments(true);
DocumentBuilderFactory builderFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = null;
try {
builder = builderFactory.newDocumentBuilder();
} catch (ParserConfigurationException e) {
e.printStackTrace();
}
TagNode tagNode = new HtmlCleaner().clean(source);
Document doc = new DomSerializer(new CleanerProperties()).createDOM(tagNode);
return doc;
} catch (ParserConfigurationException ex) {
ex.printStackTrace();
return null;
}
}
and then accessing elements via xpath like this:
String myXpathStr = "//*[@id='news-main']/div";
XPath xPath = XPathFactory.newInstance().newXPath();
NodeList articleBlocks = (NodeList)xPath.compile(myXpathStr).evaluate(doc, XPathConstants.NODESET);
hope that helps. i also agree with the other answers that id and css are faster. i've found xpath to be more powerful but I don't have a lot of experience with css paths
I search by id
, class name
and other easily identifiable elements. But speed is going to be based on things like network connection and hardware. You can alway use the HTMLDriver
, as this will be the quickest version of the driver.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.