Unable to find some html elements using Jsoup

Question

I am trying to find elements corresponding to "Apply" button in this page( https://gwlabs.taleo.net/careersection/gw_ext_career_section/jobsearch.ftl ) using Jsoup. I am tried using Document.getElementsMatchingOwnText("Apply") but it didn't return the elements. I have tried to list all the anchor elements and print their text, i couldn't see any elements with the text "Apply". Output was the following

Text of anchor is Sign In

Text of anchor is Job Search

Text of anchor is My Jobpage

Text of anchor is OK

Text of anchor is OK

Text of anchor is Submit a friend's profile

Text of anchor is View All Jobs

Text of anchor is Advanced Search

Text of anchor is Add Organization

Text of anchor is Add Location

Text of anchor is Add Job Field

Text of anchor is Cancel

Text of anchor is Add

Text of anchor is OK

Text of anchor is Clear

Text of anchor is Save this Search

Text of anchor is Single-line

Text of anchor is Save this Search

Text of anchor is Cancel

Text of anchor is Access My Saved Searches

Text of anchor is Confirm

Text of anchor is Cancel

Text of anchor is OK

Text of anchor is OK

Text of anchor is OK

Text of anchor is You can also view all available job openings.

Text of anchor is Refer a friend

Text of anchor is Previous

Text of anchor is Next

Text of anchor is OK

The same approach was giving desired results for https://healthfirst.taleo.net/careersection/hf_ext_cs/jobsearch.ftl?lang=en I am clueless as to why is this so, can somebody take a look please

Answer 1

The data you are trying to retrieve are not part of initial html, but is set by JavaScript after page is loaded. You can check it by disabling JavaScript in your browser. Jsoup only gets static html, does not execute JavaScript code.

Jsoup does not currently support JavaScript, which means that pages on which data is loaded with JavaScript will not be available when parsing using Jsoup.

If you want to get such dynamically loaded data, you can:

-Use an alternative, such as HtmlUnit, Selenium WebDriver or ui4j.

-Use the website's API, if it offers one,

-To find out from where the website loads its data, usually all you need to do is send an HTTP request somewhere to get the data as JSON.

More details can be found here: https://stackoverflow.com/tags/jsoup/info

or you can use hints from this answer: https://stackoverflow.com/a/50831894