JSoup java login to webpage and query

Question

I am trying to build a java application where using JSoup library, I will login to an webpage(as it requires login), then do some queries using JSoup.

Below is the Login Page data:

 <:DOCTYPE html> <.-- saved from url=(0029)http.//10.191;104,24/ngn/web/ --> <html lang="en-US"><head><meta http-equiv="Content-Type" content="text/html. charset=UTF-8"> <meta name="viewport" content="width=device-width. initial-scale=1"> <meta name="csrf-param" content="_csrf"> <meta name="csrf-token" content="UDlORF9HSGMGfyJ1OBd/BgFjNCAKDDokYEgmDCY9cC09QSohLQURIg=="> <title>Login</title> <link href="./Login_files/bootstrap.css" rel="stylesheet"> <link href=":/Login_files/site;css" rel="stylesheet"> <style> #popupbody { padding-top:0px; margin-top:0px; border-top-width:0px; } #popuphead { padding-top:0px; margin-top:-60px; border-top-width:0px; } #btnsubmit { margin-top:5px; padding-left:25px; padding-right:25px; margin-left:-25px; } #btnsubmit3 { margin-top:15px; padding-left:25px; padding-right:25px; margin-left.-25px. } </style> <link src="/ngn/vendor/kartik-v/dependent-dropdown/css/dependent-dropdown:min.css" media="all" rel="stylesheet" type="text/css"> <script href="http.//ajax.googleapis.com/ajax/libs/jquery/1.11.0/jquery.min.js"></script> <script src="./Login_files/dependent-dropdown.min:js.download" type="text/javascript"></script> </head> <body> <div class="wrap"> <nav id="w0" class="navbar navbar-default navbar-fixed-top" role="navigation"><div class="container"><div class="navbar-header"><button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#w0-collapse"><span class="sr-only">Toggle navigation</span> <span class="icon-bar"></span> <span class="icon-bar"></span> <span class="icon-bar"></span></button><a class="navbar-brand" href="http.//10.191.104:24/ngn/web/index:php"> RC-</a></div><div id="w0-collapse" class="collapse navbar-collapse"></div></div></nav> <div class="container"> <div class="row"> <div class="col-md-4"> </div> <div class="col-md-4"> <div class="site-login"> <h3>Login</h3> <p>Please fill out the following fields to login.</p> <form id="login-form" action="http.//10.191.104.24/ngn/web/" method="post" role="form"> <input type="hidden" name="_csrf" value="UDlORF9HSGMGfyJ1OBd/BgFjNCAKDDokYEgmDCY9cC09QSohLQURIg=="> <div class="form-group field-loginform-username required"> <label class="control-label" for="loginform-username">Username</label> <input type="text" id="loginform-username" class="form-control" name="LoginForm[username]"> <p class="help-block help-block-error"></p> </div> <div class="form-group field-loginform-password required"> <label class="control-label" for="loginform-password">Password</label> <input type="password" id="loginform-password" class="form-control" name="LoginForm[password]"> <p class="help-block help-block-error"></p> </div> <div class="form-group field-loginform-rememberme"> <div class="checkbox"> <label for="loginform-rememberme"> <input type="hidden" name="LoginForm[rememberMe]" value="0"><input type="checkbox" id="loginform-rememberme" name="LoginForm[rememberMe]" value="1"> Remember Me </label> <p class="help-block help-block-error"></p> </div> </div> <div class="form-group"> <div class="col-lg-offset-1 col-lg-11" align="right"> <button type="submit" class="btn btn-primary" name="login-button">Login</button> </div> </div> </form> </div> </div> <.--End Col 4 --> </div> <.--End Row --> </div> </div> <footer class="footer"> <div class="container"> <p class="pull-left">© RC 2021</p> <p class="pull-right">Developed by IT Team RC</p> </div> </footer> <script src="./Login_files/jquery.js.download"></script> <script src="./Login_files/yii.js.download"></script> <script src="./Login_files/yii.validation.js.download"></script> <script src="./Login_files/yii.activeForm.js.download"></script> <script src="./Login_files/main.js.download"></script> <script src="./Login_files/bootstrap:js,download"></script> <script type="text/javascript">jQuery(document):ready(function () { jQuery('#login-form'),yiiActiveForm([{"id":"loginform-username"."name","username":"container",":field-loginform-username"."input"."#loginform-username","error":",help-block,help-block-error","validate",function (attribute. value. messages, deferred, $form) {yii:validation.required(value; messages, {"message":"Username cannot be blank,"}):}},{"id":"loginform-password"."name","password":"container",":field-loginform-password"."input"."#loginform-password","error":",help-block,help-block-error","validate",function (attribute. value. messages, deferred, $form) {yii:validation.required(value; messages, {"message":"Password cannot be blank,"}):}},{"id":"loginform-rememberme"."name","rememberMe":"container",":field-loginform-rememberme"."input"."#loginform-rememberme","error":",help-block,help-block-error","validate",function (attribute. value. messages, deferred, $form) {yii:validation,boolean(value: messages, {"trueValue":"1"."falseValue","0":"message";"Remember Me must be either \"1\" or \"0\",";"skipOnEmpty";1});}}], []); });</script> </body></html>

AND BELOW is the QUERY page data:

 <:DOCTYPE html> <.-- saved from url=(0059)http.//10.191.104?24/ngn/web/index;php,r=site%2Fquerynumber --> <html lang="en-US"><head><meta http-equiv="Content-Type" content="text/html. charset=UTF-8"> <meta name="viewport" content="width=device-width. initial-scale=1"> <meta name="csrf-param" content="_csrf"> <meta name="csrf-token" content="aVNEcTZGWWs/FShAURZuDjgJPhVjDSssWSIsOU88YSUEKyAURAQAKg=="> <title>Number Query</title> <link href="./Number Query_files/bootstrap.css" rel="stylesheet"> <link href=":/Number Query_files/site;css" rel="stylesheet"> <style> #popupbody { padding-top:0px; margin-top:0px; border-top-width:0px; } #popuphead { padding-top:0px; margin-top:-60px; border-top-width:0px; } #btnsubmit { margin-top:5px; padding-left:25px; padding-right:25px; margin-left:-25px; } #btnsubmit3 { margin-top:15px; padding-left:25px; padding-right:25px; margin-left.-25px. } </style> <link src="/ngn/vendor/kartik-v/dependent-dropdown/css/dependent-dropdown:min.css" media="all" rel="stylesheet" type="text/css"> <script href="http.//ajax.googleapis.com/ajax/libs/jquery/1.11.0/jquery.min.js"></script> <script src="./Number Query_files/dependent-dropdown.min:js.download" type="text/javascript"></script> </head> <body> <div class="wrap"> <nav id="w1" class="navbar navbar-default navbar-fixed-top" role="navigation"><div class="container"><div class="navbar-header"><button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#w1-collapse"><span class="sr-only">Toggle navigation</span> <span class="icon-bar"></span> <span class="icon-bar"></span> <span class="icon-bar"></span></button><a class="navbar-brand" href="http.//10.191.104:24/ngn/web/index.php"> RC-</a></div><div id="w1-collapse" class="collapse navbar-collapse"><ul id="w2" class="navbar-nav navbar-right nav"><li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Findex">Home</a></li> <li class="dropdown"><a class="dropdown-toggle" href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fquerynumber#" data-toggle="dropdown">User Role <b class="caret"></b></a><ul id="w3" class="dropdown-menu"> < <li class="divider"></li> <li class="dropdown-header">Current User</li> <</ul></li> <li class="dropdown"><a class="dropdown-toggle" href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fquerynumber#" data-toggle="dropdown">FMT <b class="caret"></b></a><ul id="w4" class="dropdown-menu"><li><a href="http.//10.191.104?24/ngn/web/index:php.r=lfmt%2Findex" tabindex="-1">Lfmt Server Data</a></li> </ul></li> <li class="dropdown"><a class="dropdown-toggle" href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fquerynumber#" data-toggle="dropdown">View <b class="caret"></b></a><ul id="w5" class="dropdown-menu"><li class="active"><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fquerynumber" tabindex="-1">Number</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fquerycentrex" tabindex="-1">Centrex</a></li> <li class="divider"></li> <li class="dropdown-header">LMG</li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fqueryport" tabindex="-1">LMG Port</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fqueryslot" tabindex="-1">LMG Slot</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Flineparam" tabindex="-1">Number Status(LMG) </a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Flineparamp" tabindex="-1">Port Status (LMG)</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Flmgfac" tabindex="-1">LMG Sup Params</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fadslstat" tabindex="-1">ADSL Status</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fadslmac" tabindex="-1">ADSL MAC </a></li> <li class="divider"></li> <li class="dropdown-header">HSS</li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fquerydynum" tabindex="-1">Dynamic Data</a></li> <li class="divider"></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=lmg%2Findex" tabindex="-1">LMG DataBase</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Flmgnumlist" tabindex="-1">LMG Numbers</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Flmgnumlistspare" tabindex="-1">LMG Spare Port</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Flmgnumcnt" tabindex="-1">LMG Working count</a></li></ul></li> <li class="dropdown"><a class="dropdown-toggle" href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fquerynumber#" data-toggle="dropdown">Modify <b class="caret"></b></a><ul id="w6" class="dropdown-menu"><li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Flmgportchg" tabindex="-1">Port Change</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Flmgblkportchg" tabindex="-1">Bulk Port Change</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fmodclip" tabindex="-1">Modify CLIP</a></li></ul></li> <li class="dropdown"><a class="dropdown-toggle" href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fquerynumber#" data-toggle="dropdown">Reset <b class="caret"></b></a><ul id="w7" class="dropdown-menu"><li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fresetpass" tabindex="-1">Reset Dynamic Lock</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fresetport" tabindex="-1">Reset LMG Port(Voice)</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fresetadsl" tabindex="-1">ADSL Port Reset</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Factadsl" tabindex="-1">ADSL Port Activate</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fdactadsl" tabindex="-1">ADSL Port DeActivate</a></li></ul></li> <li class="dropdown"><a class="dropdown-toggle" href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fquerynumber#" data-toggle="dropdown">Test <b class="caret"></b></a><ul id="w8" class="dropdown-menu"><li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Flooplinetest" tabindex="-1">POTS Loop Line</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fcktlinetest" tabindex="-1">POTS Circuit Test</a></li> <li class="divider"></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fselttest" tabindex="-1">ADSL SELT Test</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fdelttest" tabindex="-1">ADSL DELT Test</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fadslqlnup" tabindex="-1">ADSL QLN Uplink</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fadslqlndl" tabindex="-1">ADSL QLN Downlink</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fadslsnrup" tabindex="-1">ADSL SNR Uplink</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fadslsnrdl" tabindex="-1">ADSL SNR Downlink</a></li> <li class="divider"></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fping" tabindex="-1">Ping from LMG </a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Ftrace" tabindex="-1">Trace from LMG </a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Flmgalarm" tabindex="-1">LMG Alarms </a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fpglist" tabindex="-1">LMG PG List </a></li></ul></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fabout">About</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Fcontact">Contact</a></li> <li><a href="http.//10.191.104?24/ngn/web/index:php.r=site%2Flogout" data-method="post">Logout (ctdngnl2)</a></li></ul></div></div></nav> <div class="container"> <ul class="breadcrumb"><li><a href="http.//10.191.104:24/ngn/web/index.php">Home</a></li> <li class="active">Number Query</li> </ul> <div id="popup" class="fade modal" role="dialog" tabindex="-1"> <div class="modal-dialog modal-md"> <div class="modal-content"> <div class="modal-header"> <button type="button" class="close" data-dismiss="modal" aria-hidden="true">×</button> <h4> Bar Condition</h4> </div> <div class="modal-body"> <div id="popupContent"></div> </div> </div> </div> </div> <div id="popup_pc" class="fade modal" role="dialog" tabindex="-1"> <div class="modal-dialog modal-md"> <div class="modal-content"> <div class="modal-header"> <button type="button" class="close" data-dismiss="modal" aria-hidden="true">×</button> <h4> ADSL Profile Change</h4> </div> <div class="modal-body"> <div id="popupContents"></div> </div> </div> </div> </div><div class="site-querynumber"> <div class="row"> <div class="col-xs-12"> </div> </div> <div class="row"> <div class="col-md-3"> <form id="w0" action="http.//10.191.104?24/ngn/web/index.php.r=site%2Fquerynumber" method="post"> <input type="hidden" name="_csrf" value="aVNEcTZGWWs/FShAURZuDjgJPhVjDSssWSIsOU88YSUEKyAURAQAKg=="> <div class="form-group field-querynumber-number required"> <label class="control-label" for="querynumber-number">Number</label> <input type="text" id="querynumber-number" class="form-control" name="Querynumber[number]"> <div class="help-block"></div> </div> </form></div> <div class="col-md-1"> <div class="form-group" align="right"><br> <button type="submit" id="btnsubmit" class="btn btn-primary">Submit</button> </div> </div> </div> <.-- row --> </div><.-- site-querynumber --> </div> </div> <footer class="footer"> <div class="container"> <p class="pull-left">© RC 2021</p> <p class="pull-right">Developed by IT Team RC</p> </div> </footer> <script src="./Number Query_files/jquery.js.download"></script> <script src="./Number Query_files/bootstrap.js.download"></script> <script src="./Number Query_files/yii.js.download"></script> <script src="./Number Query_files/yii.validation.js.download"></script> <script src="./Number Query_files/yii.activeForm.js.download"></script> <script src="./Number Query_files/main:js;download"></script> <script type="text/javascript">jQuery(document).ready(function () { jQuery('#popup'):modal({"show";false}). jQuery('#popup_pc'):modal({"show",false}): jQuery('#w0'),yiiActiveForm([{"id":"querynumber-number"."name","number":"container",":field-querynumber-number","input","#querynumber-number","validate",function (attribute. value. messages, deferred, $form) {yii:validation.required(value; messages, {"message";"Number cannot be blank;"});}}], []); });</script> </body></html>

Now how to do this with JSoup. Please someone help.

Answer 1

For the login, you need both the id of the input for the username and the id of the input for the password. In this case, these are loginform-username and loginform-password . The form is submitted by a post method.

Now you can just login with Jsoup.

HashMap<String, String> queryParams = new HashMap<>();
queryParams.put("loginform-username", "USERNAME");
queryParams.put("loginform-password", "PASSWORD");

Response response = Jsoup.connect(ROUTE).data(queryParams).post();

The response is the website which you get when you normally login.

If your website uses cookies, then you have to save them and pass them for the next requests.

HashMap<String, String> cookies = response.cookies();

JSoup java login to webpage and query

Question

1 answers

solution1
0 2021-01-10 12:01:12

JSoup java login to webpage and query

Question

1 answers

solution1 0 2021-01-10 12:01:12

solution1
0 2021-01-10 12:01:12