Python3 请求代理 (Socks5)

Question

Edit: I don't know if this is a thing, but often this happens when i've been testing a lot and I have a lot of python scripts running. Usually none active at the same time but I'll have ran a few of them in a short space of time. I don't know if that's why teh error starts happening.

Ok here's my dilemma. I've got a NordVPN account and I want to randomly loop through the ip's for the requests i'm making to google. This works fine.. sometimes. For some reason after a few request pings my program starts slowing down and then I start getting 'Max Connection Exceeded' errors and as far as I can tell the ip's are fine.

I started thinking that google was blocking the ip but then I request to other website and the same error continues.. I started thinking that maybe the proxy server only allows a certain amount of requests in a certain time frame so I tried some free proxies but I get the same errors again.. Either way if I leave it alone for a while it seems to work fine and then a few dozen pings later and the errors start again.

The only think I can think of and not sure how I can test or even if it's possible to test remotely (work router) is that maybe some open connections remain open and affects my code and then when the backlog of open connections are thrown out I can resume as normal. I started looking for if I needed to "close" my connection after my get request but it doesn't seem necessary (although I wasn't even sure what I was looking for).

This is one version of my code, I've tried without sessions, I've tried a different way of writing the sessions. All seem to work in the same way:

import requests, random

username = 'xxx'
password = 'yyy'

with open('proxies.txt', 'r') as p:
    proxy_lost = p.read().splitlines()
with open('data.txt', 'r') as d:
    data = d.read().splitlines()

for i in data:
    result = None
    while result is None:
        try:
            proxy = proxy_list[random.randint(0,len(proxy_list)-1)] + '.nordvpn.com'
            prox = 'socks5://{}:{}@{}:{}'.format(username, password, proxy, 1080) #I've also tried socks5h

            with requests.Session() as r:
                r.proxies['http': prox]
                r.proxies['https': prox]

                result = r.get('http://icanhazip.com')
                print(result.text.strip())

If anyone has any idea whatsoever any help is appreciated. I've reached a point where i'm struggling to think of new ideas to try.

This is an example of one of the errors I get during this whole process:

Traceback (most recent call last): File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\site-packages\socks.py", line 809, in connect negotiate(self, dest_addr, dest_port) File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\site-packages\socks.py", line 444, in _negotiate_SOCKS5 self, CONNECT, dest_addr) File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\site-packages\socks.py", line 503, in _SOCKS5_request raise SOCKS5AuthError("SOCKS5 authentication failed") socks.SOCKS5AuthError: SOCKS5 authentication failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\contrib\socks.py", line 88, in _new_conn **extra_kw File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\site-packages\socks.py", line 209, in create_connection raise err File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\site-packages\socks.py", line 199, in create_connection sock.connect((remote_host, remote_port)) File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\site-packages\socks.py", line 47, in wrapper return function(*args, **kwargs) File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\site-packages\socks.py", line 814, in connect raise GeneralProxyError("Socket error", error) socks.GeneralProxyError: Socket error: SOCKS5 authentication failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connectionpool.py", line 600, in urlopen chunked=chunked) File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connectionpool.py", line 354, in _make_request conn.request(method, url, **httplib_request_kw) File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1244, in request self._send_request(method, url, body, headers, encode_chunked) File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1290, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1239, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1026, in _send_output self.send(msg) File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 966, in send self.connect() File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connection.py", line 181, in connect conn = self._new_conn() File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\contrib\socks.py", line 110, in _new_conn "Failed to establish a new connection: %s" % error urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: SOCKS5 authentication failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\adapters.py", line 449, in send timeout=timeout File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\connectionpool.py", line 638, in urlopen _stacktrace=sys.exc_info()[2]) File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\site-packages\urllib3\util\retry.py", line 399, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: SOCKSHTTPConnectionPool(host='icanhazip.com', port=80): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: SOCKS5 authentication failed'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users*\Documents_Scripts\Find Proxies\FindProxies.py", line 23, in result = requests.get(' http://icanhazip.com ', proxies=proxies) File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\api.py", line 75, in get return request('get', url, params=params, **kwargs) File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\api.py", line 60, in request return session.request(method=method, url=url, **kwargs) File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\sessions.py", line 533, in request resp = self.send(prep, **send_kwargs) File "C:\Users*\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\sessions.py", line 646, in send r = adapter.send(request, **kwargs) File "C:\Users*\AppData\Local\Programs\P ython\Python37\lib\site-packages\requests\adapters.py", line 516, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: SOCKSHTTPConnectionPool(host='icanhazip.com', port=80): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: SOCKS5 authentication failed'))

Answer 1

Are you also closing the sessions again after use?

Try with:

r = requests.session(config={'keep_alive': False})

Inspired by: python-requests-close-http-connection