如何使用JSoup从html网页中的表格中提取特定行数据

Question

I need to extract specific row data from a table from a html web page based upon a particular string present in the row.

Basically, I need to check if a particular text value is present in the first column of the row and if it does , then I need to extract the row data.

I am trying to parse HTML using jsoup . This is my first time working with jsoup . I'm getting able to get data of all the rows but not getting able to filter it as required.

To be more specific, here is a part of inspect element data(just one of the rows of many)

<tr rr="8"><td id="ANALYSIS1_ia_pt_cid1x7" bitp="h2" bict="BIDD" biddtp="15" rowspan="9" urrowspan="9" tp="C" cc="2" rr="8" align="left" class="urST3TD urCursorClickable urSTSHL2" style="border-left:0px;border-top:0px;vertical-align:top;height:189px;padding:2px 6px 2px 6px ;"><span id="ANALYSIS1_ia_1x8_tv" ct="TV" class="urTxtStd" style="white-space:nowrap;">**Author Action required**</span></td><td id="ANALYSIS1_ia_pt_cid2x7" bitp="" bict="BIDD" biddtp="15" colspan="2" tp="C" cc="3" rr="8" align="left" class="urST3TD urCursorClickable urSubtot" style="border-left:0px;border-top:0px;height:21px;padding:2px 6px 2px 6px ;"><table cellspacing="0" cellpadding="0" id="ANALYSIS1_ia_mc2_unid0" ct="ML" class="urMatrixLayout" border="0" style="border-collapse:collapse;"><tbody class="urLinStd"><tr><td valign="top" class="urLayoutPadless"><img id="ANALYSIS1_ia_mc35" src="/com.sap.portal.design.urdesigndata/themes/portal/cp_redesign/common/treeview/ico12_treeleaf.gif" style="border-width:0px;width:;height:" alt=""></td><td valign="top" class="urLayoutPadless"><span id="ANALYSIS1_ia_2x8_tv" ct="TV" class="urTxtStd" style="white-space:nowrap;">Result</span></td></tr></tbody></table></td><td id="ANALYSIS1_ia_pt_cid4x7" bitp="" bict="BIDD" biddtp="10" tp="C" cc="5" rr="8" align="right" class="urST3TD urSubtot" style="border-left:0px;border-top:0px;height:21px;padding:2px 6px 2px 6px ;"><span id="ANALYSIS1_ia_4x8_tv" ct="TV" style="margin:0px;"></span></td><td id="ANALYSIS1_ia_pt_cid5x7" bitp="" bict="BIDD" biddtp="10" tp="C" cc="6" rr="8" align="right" class="urST3TD urSubtot" style="border-left:0px;border-top:0px;height:21px;padding:2px 6px 2px 6px ;"><span id="ANALYSIS1_ia_5x8_tv" ct="TV" style="margin:0px;"></span></td><td id="ANALYSIS1_ia_pt_cid6x7" bitp="" bict="BIDD" biddtp="10" tp="C" cc="7" rr="8" align="right" class="urST3TD urSubtot" style="border-left:0px;border-top:0px;height:21px;padding:2px 6px 2px 6px ;"><span id="ANALYSIS1_ia_6x8_tv" ct="TV" style="margin:0px;"></span></td><td id="ANALYSIS1_ia_pt_cid7x7" bitp="" bict="BIDD" biddtp="10" tp="C" cc="8" rr="8" align="right" class="urST3TD urSubtot" style="border-left:0px;border-top:0px;height:21px;padding:2px 6px 2px 6px ;"><span id="ANALYSIS1_ia_7x8_tv" ct="TV" style="margin:0px;"></span></td><td id="ANALYSIS1_ia_pt_cid8x7" bitp="" bict="BIDD" biddtp="10" tp="C" cc="9" rr="8" align="right" class="urST3TD urSubtot" style="border-left:0px;border-top:0px;height:21px;padding:2px 6px 2px 6px ;"><span id="ANALYSIS1_ia_8x8_tv" ct="TV" style="margin:0px;"></span></td><td id="ANALYSIS1_ia_pt_cid9x7" bitp="" bict="BIDD" biddtp="10" tp="C" cc="10" rr="8" align="right" class="urST3TD urSubtot" style="border-left:0px;border-top:0px;height:21px;padding:2px 6px 2px 6px ;"><span id="ANALYSIS1_ia_9x8_tv" ct="TV" style="margin:0px;"></span></td><td id="ANALYSIS1_ia_pt_cid10x7" bitp="" bict="BIDD" biddtp="10" tp="C" cc="11" rr="8" align="right" class="urST3TD urSubtot" style="border-left:0px;border-top:0px;height:21px;padding:2px 6px 2px 6px ;"><span id="ANALYSIS1_ia_10x8_tv" ct="TV" style="margin:0px;"></span></td><td id="ANALYSIS1_ia_pt_cid11x7" bitp="" bict="BIDD" biddtp="10" tp="C" cc="12" rr="8" align="right" class="urST3TD urSubtot" style="border-left:0px;border-top:0px;height:21px;padding:2px 6px 2px 6px ;"><span id="ANALYSIS1_ia_11x8_tv" ct="TV" style="margin:0px;"></span></td><td id="ANALYSIS1_ia_pt_cid13x7" bitp="" bict="BIDD" biddtp="10" tp="C" cc="14" rr="8" align="right" class="urST3TD urSubtot" style="border-left:0px;border-top:0px;height:21px;padding:2px 6px 2px 6px ;"><span id="ANALYSIS1_ia_13x8_tv" ct="TV" class="urTxtStd" style="white-space:nowrap;">1</span></td><td id="ANALYSIS1_ia_pt_cid14x7" bitp="" bict="BIDD" biddtp="10" tp="C" cc="15" rr="8" align="right" class="urST3TD urSubtot" style="border-left:0px;border-top:0px;height:21px;padding:2px 6px 2px 6px ;"><span id="ANALYSIS1_ia_14x8_tv" ct="TV" style="margin:0px;"></span></td><td id="ANALYSIS1_ia_pt_cid15x7" bitp="" bict="BIDD" biddtp="10" tp="C" cc="16" rr="8" align="right" class="urST3TD urSubtot" style="border-left:0px;border-top:0px;height:21px;padding:2px 6px 2px 6px ;"><span id="ANALYSIS1_ia_15x8_tv" ct="TV" class="urTxtStd" style="white-space:nowrap;">1</span></td><td id="ANALYSIS1_ia_pt_cid16x7" bitp="" bict="BIDD" biddtp="10" tp="C" cc="17" rr="8" align="right" class="urST3TD urSubtot" style="border-left:0px;border-top:0px;height:21px;padding:2px 6px 2px 6px ;"><span id="ANALYSIS1_ia_16x8_tv" ct="TV" class="urTxtStd" style="white-space:nowrap;">62</span></td><td id="ANALYSIS1_ia_pt_cid17x7" bitp="" bict="BIDD" biddtp="10" tp="C" cc="18" rr="8" align="right" class="urST3TD urSubtot" style="border-left:0px;border-top:0px;height:21px;padding:2px 6px 2px 6px ;"><span id="ANALYSIS1_ia_17x8_tv" ct="TV" class="urTxtStd" style="white-space:nowrap;">42</span></td><td id="ANALYSIS1_ia_pt_cid18x7" bitp="" bict="BIDD" biddtp="10" tp="C" cc="19" rr="8" align="right" class="urST3TD urSubtot" style="border-left:0px;border-top:0px;height:21px;padding:2px 6px 2px 6px ;"><span id="ANALYSIS1_ia_18x8_tv" ct="TV" class="urTxtStd" style="white-space:nowrap;">**106**</span></td><td id="ANALYSIS1_ia_pt_cid12x7" bitp="" bict="BIDD" biddtp="10" tp="C" cc="13" rr="8" align="right" class="urST3TD urSubtot" style="border-left:0px;border-top:0px;height:21px;padding:2px 6px 2px 6px ;"><span id="ANALYSIS1_ia_12x8_tv" ct="TV" style="margin:0px;"></span></td></tr>

Here , if a row contains the text " Author Action required " , I need to extract the data of the last column in the same row.

Thanks!

Answer 1

to get the "Author Action required" field using JSOUP you need to do this:

Document document = Jsoup.connect(URL).userAgent("Mozilla/5.0").get();
Elements elements = document.select("tr > td > span");

for (Element element : elements) {
    if (element.getText().equalsIgnoreCase("Author Action required")) {
        Elements elements = document.select(YOUR_NEXT_QUERY);
        ...
    }
}

What you need to do is getting te HTML tags that you want to loop over and check it the text value contained is equal to your desired text. And then do that again to retrieve the element you want in case the first condition is true.

The .getText() function gets the text contained between HTML tags as for example:

<tr id="1"><td>hi</td></tr>

That will return "hi". So if you need anything else related to Web Scraping feel free to ask me!

Hope it helps you!