Python: Make HTTP Errors (500) Not Stop Script

Question

This is the basic part of my code that I need help with. Note I learned python like last week. I don't understand try and exceptions and I know that what I need for this, so if anyone could help that would be great.

url = 'http://google.com/{0}/{1}'.format(variable, variable1)
site = urllib.request.urlopen(url)

That's not the real website but you get the idea. Now I'm running a loop over 5 times per item, then running around 20 different items. So it goes to say google.com/spiders/ (runs 5 times with diff types of spiders) google.com/dogs/ (runs 5 times with diff types of dogs)etc.

Now the 2nd variable is the same on like 90% of the items I'm looping over, but 1 or 2 of them have some of the "types" but not others. So I get an http error 500 because that site doesn't exist. How do I make it basically skip that. Its not something else, I know error 500 isn't the right error I believe, but I know the pages for those items don't exist. So how do I set this up so that it just skips that one if it gets any error.

Answer 1

You can use a try/except block in your loop, like:

try:
    url = 'http://google.com/{0}/{1}'.format(variable, variable1)
    site = urllib.request.urlopen(url)
except Exception, ex:
    print "ERROR - " + str(ex)

You can also just catch specific exceptions - the above code would catch any exception (such as a bug in your code, not a network error)

See here for more: https://wiki.python.org/moin/HandlingExceptions