Selenium ChromeDriverManager Webdriver Install not updating through proxy

Question

. 。 Answers to this question are eligible for a +50 reputation bounty. Kiran Jain is looking for an . 。

I need to access a website using selenium and Chrome using python. Below is shortened version of my code.

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.service import Service

# PROXY='https://myproxy.com:3128'
PROXY = 'https://username:password@myproxy.com:3128'
proxyuser='username' #this is the proxy user name used in above string
proxypwd='password' #this is the proxy password used in above string


chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=%s' % PROXY)
chrome_options.add_argument("ignore-certificate-errors");

chrome = webdriver.Chrome(options=chrome_options,service=Service(ChromeDriverManager().install()))

chrome.get("https://www.google.com")
while True:
    print('ok')

I am behind a corporate proxy server which requires authentication. I am not sure how to pass login credentials and proxy settings for Install of chromedriver When above code is run without proxy, it works as expected. But it gives error as follows when run using proxy connection:

[WDM] - ====== WebDriver manager ======
[WDM] - Current google-chrome version is 105.0.5195
[WDM] - Get LATEST chromedriver version for 105.0.5195 google-chrome
Traceback (most recent call last):
  File "D:\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "D:\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "D:\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 1040, in _validate_conn
    conn.connect()
  File "D:\Python\Python39\lib\site-packages\urllib3\connection.py", line 414, in connect
    self.sock = ssl_wrap_socket(
  File "D:\Python\Python39\lib\site-packages\urllib3\util\ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "D:\Python\Python39\lib\site-packages\urllib3\util\ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "D:\Python\Python39\lib\ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "D:\Python\Python39\lib\ssl.py", line 1040, in _create
    self.do_handshake()
  File "D:\Python\Python39\lib\ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Python\Python39\lib\site-packages\requests\adapters.py", line 489, in send
    resp = conn.urlopen(
  File "D:\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 785, in urlopen
    retries = retries.increment(
  File "D:\Python\Python39\lib\site-packages\urllib3\util\retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "D:\Python\Python39\lib\site-packages\urllib3\packages\six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "D:\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "D:\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "D:\Python\Python39\lib\site-packages\urllib3\connectionpool.py", line 1040, in _validate_conn
    conn.connect()
  File "D:\Python\Python39\lib\site-packages\urllib3\connection.py", line 414, in connect
    self.sock = ssl_wrap_socket(
  File "D:\Python\Python39\lib\site-packages\urllib3\util\ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "D:\Python\Python39\lib\site-packages\urllib3\util\ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "D:\Python\Python39\lib\ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "D:\Python\Python39\lib\ssl.py", line 1040, in _create
    self.do_handshake()
  File "D:\Python\Python39\lib\ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\eclipse-workspace\Essential Training\test.py", line 17, in <module>
    chrome = webdriver.Chrome(options=chrome_options,service=Service(ChromeDriverManager().install()))
  File "D:\Python\Python39\lib\site-packages\webdriver_manager\chrome.py", line 37, in install
    driver_path = self._get_driver_path(self.driver)
  File "D:\Python\Python39\lib\site-packages\webdriver_manager\core\manager.py", line 29, in _get_driver_path
    binary_path = self.driver_cache.find_driver(driver)
  File "D:\Python\Python39\lib\site-packages\webdriver_manager\core\driver_cache.py", line 95, in find_driver
    driver_version = driver.get_version()
  File "D:\Python\Python39\lib\site-packages\webdriver_manager\core\driver.py", line 42, in get_version
    self.get_latest_release_version()
  File "D:\Python\Python39\lib\site-packages\webdriver_manager\drivers\chrome.py", line 44, in get_latest_release_version
    resp = self._http_client.get(url=latest_release_url)
  File "D:\Python\Python39\lib\site-packages\webdriver_manager\core\http.py", line 31, in get
    resp = requests.get(url=url, verify=self._ssl_verify, **kwargs)
  File "D:\Python\Python39\lib\site-packages\requests\api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "D:\Python\Python39\lib\site-packages\requests\api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "D:\Python\Python39\lib\site-packages\requests\sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "D:\Python\Python39\lib\site-packages\requests\sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "D:\Python\Python39\lib\site-packages\requests\adapters.py", line 547, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

Bypassing proxy is not possible because of corporate policies

request("get", url, params=params, **kwargs)

this line in errors i believe is the issue. it seems the driver is not able to determine the new version available on net as the request function here should have proxy settings - i am not able to see the **kwargs values, ideally - i believe, **kwargs should have proxy argument. - i am not sure but.

Answer 1

I have modified the code as follows. essentially i have now used custom download manager and this is worling

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from webdriver_manager.core.download_manager import WDMDownloadManager
from webdriver_manager.core.http import HttpClient
from selenium.webdriver.chrome.service import Service
from requests import Response
import urllib3
import requests
import os
os.environ['WDM_SSL_VERIFY'] = '0'
capabilities = webdriver.DesiredCapabilities.CHROME
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

PROXY = "https://username:password@myproxy.com:3128"
proxyuser='username'
proxypwd='password'
opt = webdriver.ChromeOptions()
opt.add_argument('--proxy-server=%s' % PROXY)
opt.add_argument("ignore-certificate-errors")

class CustomHttpClient(HttpClient):
    def get(self, url, params=None) -> Response:
        proxies={'http': 'http://username:password@myproxy.com:3128',
         'https': 'http://username:password@myproxy.com:3128',
         }
        return requests.get(url, params,proxies=proxies,verify=False)

http_client = CustomHttpClient()
download_manager = WDMDownloadManager(http_client)
chrome = webdriver.Chrome(service=Service(ChromeDriverManager(download_manager=download_manager).install()),options=opt)

chrome.get("https://www.google.com")
while True:
    print('ok')