Serve any external web page from my own server using node & express

Question

I want to allow users to manipulate the DOM of another web page (let's say www.google.com for that matter).

Obviously, I don't have any intent to actually change anything on the original web page, but only to allow visual changes that seem as if they are on that website.

I started by using an iframe , but there is no way to manipulate the DOM for a cross-domain iframe . Therefore, I thought about serving that web page from my own server, and my own domain , so there would not be any cross-domain issues.

This is what I got on my server:

app.get('/showsite', (req, res) => {
  const url = 'https://www.google.com';
  request(url, (err, resp, html) => {
    if (!err) {
        res.send(html);
    }
})}

What I do here, is actually receiving the webpage using GET request, and then sending back the html response to the client.

The web page is indeed shown to the client, but the page is all messed up - relative paths are wrong, styling, encoding etc.

What should be the implementation to get it right?

Answer 1

Scraping a third party website, altering and re-displaying it's content is possible as long as you are able to pull in and host ALL of the assets that make up that website. In your case, the google.com landing page has many images that would need to retrieved and then served from your own server in your own domain.

1. You would need to use a tool like jsdom to scrape and traverse through all the sub-assets of the target website.
2. Using a tool like jsdom to download all the sub-resources, you need to copy and save all those assets locally on your server. The directory path you save them under must match the exact path of the original.
3. Once you have all the sub-resources (css, images, scripts, fonts), you can manipulate the original html and serve your new content.