需要通过Mechanize + BeautifulSoup（Python）启用Javascript的抓取网站

Question

So.. i got this site I am tryign to scrape, but as I understand lack of support of mechanize for .js, and a stuborn site that requires javascript enabled browser is not a good mix...

I am looking for ideas, on how to do this...

URL : https://members.iracing.com/membersite/login.jsp

Answer 1

Depending on what you need to do, you could use webkit to parse the page, which will allow you to get the final html after the javascript has been executed. You could then use any decent html parser, beautifulsoup for example, to do the rest.

Answer 2

使用JavaScript，我将Chickenfoot用于简单的网站，将Webkit用于更复杂的网站。