自动执行网络任务？

Question

I play on chess.com and I'd like to download a history of my games. Unfortunately, they don't make it easy: I can access 100 pages of 50 games one at a time, click "Select All" and "Download" and then they e-mail it to me.

Is there a way to write a script, in python or another language, that helps me automate any part of the process? Something that simulates clicking a link? Is Capybara useful for things like this outside of unit testing? Selenium?

I don't have much experience with web development yet. Thanks for your help!

Answer 1

You may want to check out CasperJS. I use Python to fire CasperJS scripts to do web scraping and return data to Python to parse further or store to a database etc...

Python itself has BeautifulSoup and Mechanize but the combination is not great with Ajax based sites.

Python and CasperJS is perfect.

Answer 2

Selenium could be a good candidate if you want to do something like this.

Here would be some pseudo code:

foreach page_link in page_links:
  page_link.click()
  select_all.click()
  download_link.click()

Answer 3

Take a look at Helium . It should be relatively easy to achieve what you need to do using this Python library:

start_chrome("chess.com")
...
click("Select All")
click("Download")
...

etc.