HtmlUnit cannot get page to change after clicking element

Question

I'm currently trying to scrape some information from https://www.overbuff.com/heroes using HtmlUnit but I can't get the page to actually change after clicking on the selected element. I'm trying to click the elements in the rank box [All, Bronze, Silver...] so that I can change the information on the page and get the info for that specific rank, but after clicking the page stays the same. I've tried multiple ways of finding the specific element I want and I find it without an issue each time but the click never updates the page.

public static void testPageGrab() throws Exception {
    WebClient client = new WebClient(BrowserVersion.FIREFOX_60);
    client.getOptions().setJavaScriptEnabled(true);
    client.getOptions().setThrowExceptionOnScriptError(false);
    client.getOptions().setThrowExceptionOnFailingStatusCode(false);
    HtmlPage page = client.getPage("https://www.overbuff.com/heroes");
    DomNodeList<DomNode> nodes = page.querySelectorAll("span.r-none-mobile");
    HtmlPage page2 = ((HtmlElement) nodes.get(0)).click();
    client.waitForBackgroundJavaScript(5000);
    System.out.println(page2.asText());
}

The element that is returned from nodes.get(0) is the element for the "Bronze" category, yet the text printed from page2 is the same as from page1 which used the "All" category. Does anyone know why the page refuses to update? I've looked at other threads that seemed to have similar issues but the solutions that worked for them such as waiting for the javascript haven't worked for me. One thread talked about AJAX requests and using the FancyAjaxController but that also didn't have any effect for me.

Answer 1

I fear this is because the javascript fails before the site gets updated.

com.gargoylesoftware.htmlunit.ScriptException: TypeError: redeclaration of formal parameter t. ( https://www.overbuff.com/assets/static/app.js?v=8f65c97#1 )

This one is already on the list of fixes to be done in Rhino - but it is not that simple to fix.