Fetching data from another website with JSOUP

Question

Basically, I need a table with all the possible books that exist, and I don't wanna do that, because I'm a very lazy person xD. So, my question is.. can I use a site, that I have in mind, and just like cut off the rest this site(that I don't need) and leave only the search part(maybe do some kind of changes in layout)... then, make the search, find the book and store in my database only the data that make sense for me. Is that possible? I heard that JSOUP could help. So, I just want some tips. (thx for reading).

the site: http://www.isbn.bn.br/website/consulta/cadastro

Answer 1

Yes, you can do that using Jsoup, the main problem is that the URL you shared uses JavaScript so you'll need to use Selenium to force the JS execution or you can also get the book URL and parse it.

The way to parse a web using Jsoup is:

Document document = Jsoup.connect("YOUR-URL-GOES-HERE")
                         .userAgent("Mozilla/5.0")
                         .get();

The you retrieve the whole HTML in a Document so you can get any Element contained in the Element using CSS Selectors, for example, if in the HTML you want to retrieve the title of the web, you can use:

Elements elements = document.select("title");

And that for every HTML tag that you want to retrieve information from. You can check the Jsoup Doc an check some of the examples explained: Jsoup

I hope it helps you!