Login to page then parse html using JSOUP

Question

I currently have an app that takes a username & password from the user and then uses javascript commands on webview to log them in and present the page. I now want to avoid using a webview, so Im planning to use jsoup to pull the necessary parts of the page and present that using something such as a textview instead. So how can I use jsoup to execute javascript to log the user in, then scrape the html?

Answer 1

Jsoup can't execute javascript, since it doesn't include a javascript engine. What you can use is the web driver api of selenium. There is a new project now called android web driver that could be proved the right tool for you (never tried it myself).

Links:

Android web driver

Selenium web driver

Answer 2

Jsoup does not execute JavaScript. But I think you could still use jsoup in this case. It depends on how that JavaScript works. In general it'll be like doing a page submit to some URL with credentials in POST data. So you need to identify the parameter name and value for both username and password. Then you could use Jsoup to post your data to that URL with appropriate credentials.

You could try something like

Document doc = Jsoup.connect("http://www.yoururl.com/")
            .data("userName", "your username")
            .data("password", "your password")
            .post();

Then with the doc object you can parse and display your view.

Answer 3

You should first logon and then parse the html. You can use cookies for logins and re-call the cookies in your further codes, like that.

        Response res = Jsoup.connect("http://yourloginURL").data("username", "username", "password", "password").method(Method.POST).execute();
    Map<String, String> loginCookies = res.cookies();

    Document doc1 = Jsoup.connect("http://yourotherURL").cookies(loginCookies).get();