执行JavaScript链接后从网页获取HTML

Question

There is an internet page, that when you click a javascript link ( tag with javascript:... in it) loads a table. I need to get this table into my Asp.net website. There is no URL that contains the table without executing any scripts. This is what I am currently using:

public string GetFromUrl(string path)
{
    WebClient web = new WebClient();
    return web.DownloadString(path);
}

public string GetTagHTML(string html)
{
    Regex regex = new Regex("<table>(.*)</table>");
    var v = regex.Match(html);
    return v.Groups[1].ToString();
}

more info

The website I am trying to get data from is http://beitbiram.iscool.co.il/default.aspx (it's in hebrew. The link I am trying to click is one of the table titles).

The website is an asp.net website.

The function that the link calls is __doPostBack . I don't have any idea what it does, and can't find any online info about it, but this is it's code:

var theForm = document.forms['Form'];
if (!theForm) {
    theForm = document.Form;
}
function __doPostBack(eventTarget, eventArgument) {
    if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
        theForm.__EVENTTARGET.value = eventTarget;
        theForm.__EVENTARGUMENT.value = eventArgument;
        theForm.submit();
    }
}

Thanks in advance.

Answer 1

In general, the only way to get the HTML of a page after running Javascript is to run the Javascript, and that requires a browser.

The direct answer to your question, then, is to use something like Headless Chrome to spin up a browser, load the page, click the link, and export the HTML. This has historically been a massive pain to get working, although Headless Chrome is supposed to be rather less painful.

However, the javascript: link you run must get the data from somewhere in order to put it into the table, so I would strongly advise looking for that source and building the table yourself, because I certainly wouldn't want to be maintaining a website with an embedded browser if I didn't absolutely have to.