[英]python urllib2 timeout
好的,我已经在google中和这里的stackoverflow中搜索了此答案,几个小时后却没有看到执行此操作的脚本的正确答案。
在这里,我粘贴了4个假定的python工作脚本示例,以为不存在的url设置默认超时,并使用套接字和/或超时参数设置超时。
没有人可以使用,因此永远不会触发超时。
有任何想法吗?
第一个例子:
import urllib2
try:
header_s = {"User-Agent":"Mozilla/5.0 (X11; U; Linux i686) Gecko/20071127 Firefox/2.0.0.11"}
req = urllib2.Request("http://www.nonexistantdomainurl.com/notexist.php",headers = header_s)
print urllib2.urlopen(req, None, 5.0).read()
except urllib2.URLError, e:
print "Url Error: %r" % e
except Exception,e:
print "Fallo de tipo ",e
else:
print "all ok!"
第二个例子:
import urllib2
try:
response = urllib2.urlopen("http://www.nonexistantdomainurl.com/notexist.php", None, 2.5)
except urllib2.URLError, e:
print "Oops, timed out?"
阈值示例:
from urllib2 import Request, urlopen, URLError, HTTPError
import base64
req = Request('http://www.nonexistantdomainurl.com/notexist.php')
try:
response = urlopen(req,timeout=5.0)
except HTTPError, e:
print 'The server couldn\'t fulfill the request.'
print 'Error code: ', e.code
except URLError, e:
print 'We failed to reach a server.'
print 'Reason: ', e.reason
第四个例子:
import urllib2
import socket
socket.setdefaulttimeout(5)
try:
response = urllib2.urlopen("http://www.faluquito.com/equipo.php",timeout=5.0).read()
except urllib2.URLError, e:
print "Url Error: %r" % e
>>> import urllib2
>>> import time
>>> import contextlib
>>>
>>> def timeit():
... s = time.time()
... try:
... yield
... except urllib2.URLError:
... pass
... print 'took %.3f secs' % (time.time() - s)
...
>>> timeit = contextlib.contextmanager(timeit)
>>> with timeit():
... r = urllib2.urlopen('http://loc:8080', None, 2)
...
took 2.002 secs
>>> with timeit():
... r = urllib2.urlopen('http://loc:8080', None, 5)
...
took 5.003 secs
如果您的计算机具有unix程序dig,则可以识别出不存在的url,如下所示:
import logging
import subprocess
import shlex
logging.basicConfig(level = logging.DEBUG,
format = '%(asctime)s %(module)s %(levelname)s: %(message)s',
datefmt = '%M:%S')
logger = logging.getLogger(__name__)
urls = ['http://1.2.3.4',
"http://www.nonexistantdomainurl.com/notexist.php",
"http://www.faluquito.com/equipo.php",
'google.com']
nonexistent = ['63.251.179.13', '8.15.7.117']
for url in urls:
logger.info('Trying {u}'.format(u=url))
proc = subprocess.Popen(shlex.split(
'dig +short +time=1 +retry=0 {u}'.format(u = url)),
stdout = subprocess.PIPE, stderr = subprocess.PIPE)
out, err = proc.communicate()
out = out.splitlines()
logger.info(out)
if any(addr in nonexistent for addr in out):
logger.info('nonexistent\n')
else:
logger.info('success\n')
在我的机器上,这产生了:
00:57 test INFO: Trying http://1.2.3.4
00:58 test INFO: ['63.251.179.13', '8.15.7.117']
00:58 test INFO: nonexistent
00:58 test INFO: Trying http://www.nonexistantdomainurl.com/notexist.php
00:58 test INFO: ['63.251.179.13', '8.15.7.117']
00:58 test INFO: nonexistent
00:58 test INFO: Trying http://www.faluquito.com/equipo.php
00:58 test INFO: ['63.251.179.13', '8.15.7.117']
00:58 test INFO: nonexistent
00:58 test INFO: Trying google.com
00:58 test INFO: ['72.14.204.113', '72.14.204.100', '72.14.204.138', '72.14.204.102', '72.14.204.101']
00:58 test INFO: success
请注意,对于不存在的网址,dig将返回['63.251.179.13', '8.15.7.117']
。
我相信我的ISP将不存在的地址更改为63.251.179.13或8.15.7.117。 您的ISP可能会做一些不同的事情。 在这种情况下,您可能需要将nonexistent
内容更改为其他内容。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.