如何加快此脚本的速度？

Question

I am required to retrieve 8000 answers from a website for research purposes (auto filling a form and submitting it 8000 times). I wrote the below script but when I run it after 20 submits python stops working and I'm unable to get what I need. Could you please help me find the problem with my script?

from mechanize import ParseResponse, urlopen, urljoin
import urllib2
from urllib2 import Request, urlopen, URLError
import mechanize
import time

URL = "url of the website"
br = mechanize.Browser() # Creates a browser
br.set_handle_robots(False) # ignore robots
br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), max_time=1)

def fetch(val):
    br.open(URL) # Open the login page
    br.select_form(nr=0) # Find the login form
    br['subject']='question'
    br['value'] =val
    br.set_all_readonly(False)
    resp = br.submit()
    data = resp.read()
    br.reload()
    x=data.find("the answer is:")
    if x!=-1:
        ur=data[x:x+100]
        print ur

val_list =val_list # This list is available and contains 8000 different values

for i in range(0,8000):
    fetch(val_list[i])

Answer 1

Having used mechanize in the past to do a similar data-scraping kind of thing, you're almost certainly getting limited by the website as Erbureth mentioned. Usually websites have a way to monitor connections to filter out exactly the type of thing you're attempting, and for good reason.

Putting aside for a moment whatever the purpose of your script may be and moving to your question of why is doesn't work: At the very least, I would put some delays in there so you're not trying to access the site repeatedly in such a short time span. Put a few seconds of pause between calls, and maybe it will work. (Although then you'll have to let it run for hours.)