python urllib2中的代理和URL

Question

Here is my code but it is giving me some error which i an unable to solve it. Even though same code is running fine with a single url and a single proxy but it is not running for the file of proxy and urls .

import urllib2
import time 
#bangalore, boston,china

with open('urls.txt') as f:
    urls = [line.strip() for line in f]
    print "list of urls",urls
with open('proxies.txt') as proxies:
    for proxy in proxies:
        print proxy
        proxy = proxy.rstrip()
        print proxy
        proxy_handler = urllib2.ProxyHandler(proxy)
        opener = urllib2.build_opener(proxy_handler)
        urllib2.install_opener(opener)
        try:
            for url in urls:
                request=urllib2.Request(url)
                start=time.time()
                try:
                    print "from try block"
                    response=urllib2.urlopen(urls[0])
                    response.read(1)
                    ttfb = time.time() - start
                    print "Latency:", ttfb
                    print "Status Code:", response.code
                    print "Headers:", response.headers
                    print "Redirected url:", response.url  
                except urllib2.URLError as e:
                    print "From except"
                    print "Error Reason:", e.reason
                    print "Error Message:", e.message
                   # print "Redirected URL:", e.url
                except urllib2.HTTPError as e:
                    print e.reason 
        except Exception,e:
            print e

Answer 1

replace with this:

proxy = json.loads(proxy.rstrip())

(and import json)

urls.txt lines are like:

http://www.google.com

proxies.txt lines are like:

{"http" : "http://ip:port"}

also this will always refer to the first url as per my comment on your post:

response=urllib2.urlopen(urls[0])