客户端的网页抓取

Question

This is probably not the best title for this question.

So i have a nodejs application running on my server which currently uses a python script for web-scraping but i am looking at moving this to the client-side due to individual client seeing different versions (potentially unique) of the same site.

I an ideal world i would like to use javascript to get the html response from a page (what i can see in chrome by right-clicking and choosing view source) to then be processed in javascript.

However from what i have read online this does not seem to be possible. I am aware of sites that provide the response (such as anyorigin.com) that can be scraped. However, these are not really suitable for me as i need to be able to scrape what the user see's as each user can potentially see something different on the site i want to scrape. The python script i am currently using would do this but it would require the user to have python installed in order for me to be able to execute it and this cannot be guaranteed.

Apologies for the block of text.

Is there any solution to this problem ?

Answer 1

After some research and the suggestions received, i created a chrome extension using the simple guide on the Chrome Developer site and used a CORSrequest to get what i needed.

If anyone finds this question and would like help, i am happy to provide further details/assistance :)

Answer 2

I was recently trying to do something very similar, and unfortunately, as far as I know there's not a way to do this on the client-side. You may be able to do some trickery and "post" the data you need back you the server where you deal with it, but I don't imagine that will be very efficient or straight forward.

Though if you do find something, please do share.