什么称为允许模拟浏览器以实现自动化目的的库？

Question

I have an automating task in which I need to fill several forms in a site with data from word documents. For that I would need a library that emulates a browser and allows me to programatically enter a site and access html elements. What is this called? Are there examples of libraries that do this for python or clojure?

Answer 1

You have a couple of choices:

1. Mechanize
2. Selenium

There are others too, but I can't remember them off the top of my head right now (will post as and when I remember more)

Answer 2

You may wanna take a look at PhantomJS too:

PhantomJS is a headless WebKit with JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG.

If you just want to submit a form, it would probably be easier to forge a request and send it using urllib2 .

Answer 3

In nowadays Clojure , http-kit is my favorite. It just makes http interaction very easy.

; taken from github
(defn on-response [resp]
  ;; {:status 200 :body "....." :headers {:key val :key val}}
  (println resp))

;;; initialize, timeout is 40s, and default user-agent
 (http/init :timeout 40000 :user-agent "http-kit/1.1")

;;; other params :headers :proxy binary? keyify?
(http/get {:url "http://shenfeng.me" :cb on-response})

;;; other params :headers :proxy binary? keyify?
(http/post {:url "http://example/"
        :cb on-response
        :body {"name" "http-kit" "author" "shenfeng"}  :binary? true})

I have also used CasperJs and it just makes any headless browsing possible. Also, you can interact with the client side javascript while automating the browsing. The only draw back I found was that it was slightly harder to integrate all this with existing code, but as a standalone tool it was perfect. It also supports both coffescript and javascript scripting.

Look at the Quickstart to get an idea on how it works.