How to login a webpage using "Jsoup" before parsing its data?

Question

I've written java program to connect to https://www.valueresearchonline.com/funds/portfoliovr.asp?schemecode=26123 using Jsoup and parse data.

Now the website has mandated to login first before showing entire data.

https://www.valueresearchonline.com/login

Is there a way to login a website first with Jsoup and later parse it?

Answer 1

pom.xml

<dependency>
              <groupId>org.jsoup</groupId>
              <artifactId>jsoup</artifactId>
               <version>1.15.3</version>
          </dependency>

java file imports as below

import org.jsoup.Connection;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;

Working code below

 Connection.Response res = Jsoup.connect("https://www.valueresearchonline.com/login").userAgent("Mozilla")
                .data("username", "your_username", "password", "your_password")
                .method(Connection.Method.POST)
                .execute();

    System.out.println(res.statusCode());

//This will get you cookies

Map<String, String> loginCookies = res.cookies();

//Here you parse the page that you want. Put the url that you see when you have logged in

Document doc = Jsoup.connect("https://www.valueresearchonline.com/")
            .userAgent("Mozilla")
            .cookies(loginCookies)
            .get();

System.out.println(doc.baseUri());