当数据来自JavaScript时，我可以从网页上抓取数据吗？

Question

I'm not exactly sure how to phrase my question but I'll give it my best shot.

If I load up a webpage, in the HTML it executes a JavaScript file. And if I view the page source I can see the source of that JavaScript (though it's not very well formatted and hard to understand).

Is there a way to run the JavaScript from eg Python code, without going through the browser? ie if I wanted to access a particular function in that JavaScript, is there a clean way to call just that from a Python script, and read the results?

For example... a webpage displays a number that I want access to. It's not in the page source because it's a result from a JavaScript call. Is there a way to call that JavaScript from Python?

Answer 1

If you want to scrape a page with javascript in it you've got at least two options:

• Use selenium to load the page and get the node value you're interested in
• Use python-spidermonkey to leverage the javascript right from your python script and get the value you're interested in.

Answer 2

Although your question isn't very clear. I'm guessing that you are trying to access the javascript console.

In Google Chrome:

• Press F12
• Go to the 'console' tab

In Mozilla Firefox with Firebug installed:

• Open Firebug
• Go to the 'console' tab

From the console you can execute javascript query's (calling functions, accessing variables etc.).

I hope this answered your question properly.

Answer 3

I think you are talking about Obfuscate js code

You can always de-obfuscate them

There are lots of tools availaible

Here is a addon of mozilla https://addons.mozilla.org/en-us/firefox/addon/javascript-deobfuscator/

and an online tool http://jsbeautifier.org/