简体   繁体   中英

Python requests.get proxy not working?

I am a new developer, so please forgive my ignorance.

I am trying to use proxies access some webpages in Python. I have tried using both urllib2 and the requests module in conjunction with various proxies that I believe to be working. However, when I go to a site to verify that my ip is showing up as the proxy, its still showing my actual ip address and not the proxy!

That leads me to conclude that there are four possible things going on:

  1. These modules are broken. This seems highly unlikely, but possible.
  2. My code is incorrect. Highly probable, however I cannot determine specifically any errors.
  3. There is some way to detect the root IP , even if they are routing through a proxy.
  4. Something I haven't thought of.

Any help is appreciated!

import requests
import urllib2
from bs4 import BeautifulSoup

# Using requests module
proxy_dict = {"http":"http://123.45.172.115:8080"}
url = 'https://check.torproject.org/'
response = requests.get(url, proxies=(proxy_dict))
html = response.content
soup = BeautifulSoup(html)
ip = str(soup.b.text)

# Using urllib2
prox = urllib2.ProxyHandler(proxy_dict)
opener = urllib2.build_opener(prox, urllib2.HTTPHandler(debuglevel=1))
urllib2.install_opener(opener)
response = opener.open(url)

Your code seems quite correct. I suspect these sites are actually using the X_FORWARDED_FOR or other similar HTTP header value, making your option number 3 the most likely.

All your examples are certainly using my originating IP address instead of my proxy address, excepting for when I bounce through a VPN, which shows they are actually using this header.

I had a similar problem and it turned out that it was a transparent proxy and the root IP was being passed along. So option number 3.

you can test this a bit further by running a request through http://www.lagado.com/proxy-test

This shows you any transparent proxies that your request has gone through, so you can test that the proxy is actually working.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM