NameError：未定义名称“ htmltext”

Question

I obtain an error when i run this script:

import urllib.request
import urllib.parse
from bs4 import BeautifulSoup

url = "http://nytimes.com,http://nytimes.com"

urls = [url] #stack of urls to scrape
visited = [url] #historic record of urls

while len(urls) >0:
try:
    htmltext = urllib.request.urlopen(urls[0]).read()
except:
    print(htmltext)

Original scipt:

import urllib.request
import urllib.parse
from bs4 import BeautifulSoup

url = "http://nytimes.com,http://nytimes.com"

urls = [url] #stack of urls to scrape
visited = [url] #historic record of urls

while len(urls) >0:
try:
    htmltext = urllib.request.urlopen(urls[0]).read()
except:
    print(urls[0])
soup = BeautifulSoup(htmltext)

urls.pop(0)

print (soup.findAll('a',href=True))

Errors:

socket.gaierror: [Errno -2] Name or service not known

urllib.error.URLError: urlopen error [Errno -2] Name or service not known

Traceback (most recent call last):

NameError: name 'htmltext' is not defined

Answer 1

If urllib.request.urlopen() raises an exception, htmltext never gets assigned a value (so printing that value in except won't work).

As to why urlopen() is not working, make sure you are passing a valid URL.