简体繁体 English

在本地html文件上评估javascript（无需浏览器）

[英]Evaluate javascript on a local html file (without browser)

原文 2013-05-04 14:16:10 3 2 javascript/ python/ html/ screen-scraping/ eval

This is part of a project I am working on for work. 这是我正在工作的项目的一部分。

I want to automate a Sharepoint site, specifically to pull data out of a database that I and my coworkers only have front-end access to. 我想自动化一个Sharepoint网站，特别是要从我和我的同事只能访问前端的数据库中提取数据。

I FINALLY managed to get mechanize (in python) to accomplish this using Python-NTLM, and by patching part of it's source code to fix a reoccurring error. 我终于设法使用python-ntlm进行了机械化（在python中）来完成此任务，并且通过修补了部分源代码以修复重复出现的错误。

Now, I am at what I would hope is my final roadblock: Part of the form I need to submit seems to be output of a JavaScript function :| 现在，我希望最后的障碍是：我需要提交的部分表单似乎是JavaScript函数的输出：| and lo and behold... Mechanize does not support javascript. 和瞧瞧...机械化不支持javascript。 I don't want to emulate the javascript functionality myself in python because I would ideally like a reusable solution... 我不想自己在python中模拟javascript功能，因为理想情况下，我想要一个可重用的解决方案...

So, does anyone know how I could evaluate the javascript on the local html I download from sharepoint? 因此，有人知道我如何评估我从sharepoint下载的本地html上的javascript吗？ I just want to run the javascript somehow (to complete the loading of the page), but without a browser. 我只想以某种方式运行javascript（以完成页面的加载），但是没有浏览器。

I have already looked into selenium, but it's pretty slow for the amount of work I need to get done... I am currently looking into PyV8 to try and evaluate the javascript myself... but surely there must be an app or library (or anything ) that can do this?? 我已经研究过硒，但是要完成的工作量相当慢……我目前正在研究PyV8自己尝试评估javascript……但是肯定有一个应用程序或库（或任何可以做到这一点的东西？

2 个解决方案

Well, in the end I came down to the following possible solutions: 好吧，最后我得出以下可能的解决方案：

Run Chrome headless and collect the html output (thanks to koenp for the link!) 无头运行Chrome并收集html输出（感谢koenp提供的链接！）
Run PhantomJS , a headless browser with a javascript api 运行PhantomJS ，这是一个带有JavaScript API的无头浏览器
Run HTMLUnit ; 运行HTMLUnit ; same thing but for Java 同样的事情，但对于Java
Use Ghost.py , a python-based headless browser (that I haven't seen suggested anyyyywhere for some reason!) 使用Ghost.py ，这是一个基于python的无头浏览器（出于某种原因我没有看到建议的anyyyywhere！）
Write a DOM-based javascript interpreter based on Pyv8 (Google v8 javascript engine) and add this to my current "half-solution" with mechanize. 编写基于Pyv8（Google v8 javascript引擎）的基于DOM的javascript解释器，并使用机械化将此添加到我当前的“半解决方案”中。

For now, I have decided to use either use Ghost.py or my own modification of the PySide/PyQT Webkit (how ghost works) to evaluate the javascript, as apparently they can run quite fast if you optimize them to not download images and disable the GUI. 就目前而言，我决定使用Ghost.py或我对PySide / PyQT Webkit的修改（Ghost的工作方式）来评估javascript，因为如果您优化它们以免下载图像并禁用它们，显然它们可以运行得非常快。 GUI。

Hopefully others will find this list useful! 希望其他人会发现此列表有用！

Well you will need something that both understands the DOM and understand Javascript, so that comes down to a headless browser of some sort. 好吧，您将需要既能理解DOM又能理解Javascript的东西，从而可以归结为某种无头浏览器。 Maybe you can take a look at the selenium webdriver , but I guess you already did that. 也许您可以看一下selenium webdriver ，但是我想您已经做到了。 I don't hink there is an easy way of doing this without running the stuff in an actually browser engine. 我不认为没有在实际的浏览器引擎中运行内容的简便方法。