jsoup to retrieve data from web page in key value pairs

Question

please someone help me out , using jsoup how could be traverse data , the data i want to fetch is totally in JavaScript of web page , and written into key-value pairs , so there's no any html tags for that and no any img tags , any one help me out how can i traverse that to fetch my data . in jsoup i can;t find any way to traverse this data with key value pairs like in JSON. first i tried using API but this is limited option ,

edge_sidecar_to_children

under this all links i want to fetch , any body help me out from this please

See the data that i want to display

Answer 1

Jsoup is not a browser and it does not run JavaScript. So if a JavaScript needs to run to populate some HTML from JSON, you are out of luck.

Two solutions:

1) Parse the JSON within the page. Jsoup can then only be used to get the page and maybe get the raw JSON, but then you should use a specialized library for JSON parsing.

2) Use a browser, ie something like Selenium webdriver. This will start a real (headless) browser that you can control via Java. This way all JavaScript will run and you can fetch the HTML afterwards. This approach may be resource hungry and probably runs a lot slower than method 1.