Using Python to Automatically Login to a Website with a JavaScript Form

Question

I'm attempting to write a particular script that logs into a website. This specific website contains a Javascript form so I had little to no luck by making use of "mechanize".

I'm curious if there exist other solutions that I may be unaware of that would help me in my situation. If this particular question or some related variant has been asked here before, please excuse me, and I would prefer the link to this particular query. Otherwise, what are some common techniques/approaches for dealing with this issue?

Thanks.

Answer 1

I've recently been using PhantomJS for this kind of work - it's a command-line tool that allows you to run Javascript in a browser environment (based on Webkit). This allows you to do scraping and online interactions that require Javascript-enabled interfaces. There's a Python-based implementation here that's fully compatible with the API of the C++ version, or you could run either version in Python via subprocess .

Depending on what you're trying to do, another good option might be to use Selenium , which has client driver implementation in Python - it's meant for integration testing, but can do a lot of automation as long as you're okay running the Java-based Selenium Server and having the automation happen in an open browser rather than as a background process.