[英]How to use requests proxy and how to check if a proxy is working with Python requests?
我陷入了python中的requests
代理。
下面是我的代碼:
import requests
from lxml.html import fromstring
from bs4 import BeautifulSoup
url = 'https://free-proxy-list.net/'
response = requests.get(url)
parser = fromstring(response.text)
proxies = []
for i in parser.xpath('//tbody/tr')[:10]:
if i.xpath('.//td[7][contains(text(),"yes")]'):
#Grabbing IP and corresponding PORT
proxy = ":".join([i.xpath('.//td[1]/text()')[0], i.xpath('.//td[2]/text()')[0]])
proxies.append(proxy)
url = 'https://www.ebay.com'
dic_proxies = {'https:':'http://' + str(proxies[0]),
'http':'http://' + str(proxies[0])}
u_a = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.82 Safari/537.36"
page = requests.get(url, proxies=dic_proxies,headers={"USER-AGENT":u_a})
soup = BeautifulSoup(page.text, 'html.parser')
p_soup = soup.prettify()
但是,上面的代碼似乎只能從我的實際位置檢索網站,而不能使用其他代理訪問世界其他地區的網站。 我是否以正確的方式使用requests
? 還是我的代理無法正常工作? 如何檢查代理是否正常運行?
import requests
from requests.auth import HTTPBasicAuth
requests.get(url, auth=HTTPBasicAuth(<user>, <password>), stream=True, timeout=3)
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.