复杂的Web文档检索

Question

I need to retrieve a document from a website, and parse it. Problem is that:

1. The site uses both http and https protocol
2. You need to log in the site (I have a regular account)
3. From the login page, there are at least 2 redirect just to log in yourself

I managed an HTTPS connection and posted my login and pass, but I'm having troubles with cookie management and the redirect....

Answer 1

commons-httpclient会有所帮助。

Answer 2

使用类似HtmlUnit的库可能会有所帮助。