jQuery-获取HTTP请求未在获取的源中执行JavaScript

Question

My goal is to create a Chrome plug-in that fetches a webpage from a given URL and scrapes it's HTML content to find a download link to a video. The problem is that the fetched webpage runs some JavaScript to generate download links on the page. However, when I fetch the page using below jQuery code, I don't see the download URL as (possibly) the JavaScript on the fetched page hasn't executed.

$.get(link, function(response) {
    console.log($response);
    //console.log($(response).find(".videoData").attr("value"));
});

I looked online and got to know that jQuery's .load() function will also fail in this case. PhantomJS's page.open() function seems to do what I require, but I want to deploy some lightweight solution inside my Chrome extension, so I am unable to use PhantomJS.

Answer 1

You can inject a Content Script in the web page you're trying to scrape. Make sure that you enter "document_end" in the manifest so you can ensure that your script executes after the target page has finished loading.

Another solution would be to create an invisible iframe within which you'd load the target url, and then scrape the loaded code for the video links.