[英]Htmlunit : How to get page updated after ajax dom manipulation
Using HtmlUnit 2.15, we are trying to scrape a third party website. 使用HtmlUnit 2.15,我们正在尝试抓取第三方网站。 Therein is a textbox, which onblur calls a javascript function, which adds an option to a select box on the same page.
其中有一个文本框,onblur调用了javascript函数,该函数将选项添加到同一页面上的选择框。
With Htmlunit, I am able to successfully fire the onblur event, but how do I get handle to the "changed" page, which has the newly added option element? 使用Htmlunit,我能够成功触发onblur事件,但是如何获取具有新添加的option元素的“ changed”页面的句柄?
Code snippet: 程式码片段:
final HtmlPage page = webClient.getPage(myUrl);
HtmlSelect selectDropDown = (HtmlSelect)page.getElementByName(selectname);
List<HtmlOption> options = clickThis.getOptions(); // returns 4 options
HtmlTextInput myTextBox = page.getElementByName(textboxname);
myTextBox.setValueAttribute("myText");
myTextBox.fireEvent(Event.TYPE_BLUR);
// Now how do I get the "updated" page? //现在如何获取“已更新”页面? It should have 5 options
它应该有5个选择
You need to wait until the javascript has changed the page. 您需要等待javascript更改页面。 My experience is that this may take a while.
我的经验是,这可能需要一段时间。 Especially if calling of the server is part of it.
尤其是在调用服务器的过程中。
So far My attempt is to poll the page until something changed the way I expect it to. 到目前为止,我的尝试是轮询页面,直到某些事情改变了我期望的方式。
Here is a method that waits for a given text to appear on the page; 这是一种等待给定文本显示在页面上的方法。
private static final int AJAX_MAX_TRIES_SECONDS = 30;
/**
* Waits until the given 'text' appeared or throws an
* WaitingForAjaxTimeoutException if the 'text' does not appear before we timeout.
* @param page
* @param text The text which indicates that ajax has finished updating the page
* @param waitingLogMessage Text for the log-output. Should indicate where in the code we are, and what are we waiting for
* @throws WaitingForAjaxTimeoutException
*/
public static void waitForAjaxCallWaitUntilTextAppears(//
@Nonnull final HtmlPage page, //
@Nonnull final String text,//
@Nonnull final String waitingLogMessage) throws WaitingForAjaxTimeoutException {
LOGGER.debug("_5fd3fc9247_ waiting for ajax call to complete ... [" + waitingLogMessage + "]");
final StringBuilder waitingdots = new StringBuilder(" ");
for (int i = 0; i < AJAX_MAX_TRIES_SECONDS; i++) {
if (page.asText().contains(text)) {
waitingdots.append(" ajax has finished ['").append(text).append("' appeared]");
LOGGER.debug("_8cd5a34faf_ " + waitingdots);
return;
}
waitingdots.append('.');
wait(page);
}
LOGGER.debug("_de5091bc9e_ "
+ waitingdots.append(" ajax timeout ['").append(text).append("' appeared NOT]").toString());
LOGGER.debug("_f1030addf1_ page source:\n" + page.asXml());
throw new WaitingForAjaxTimeoutException();
}
Also make sure that javascript is enabled. 还要确保启用了javascript。 (which is the default):
(默认设置):
webClient.getOptions().setJavaScriptEnabled(true);
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.