How to configure special proxy settings for a remote selenium webdrive with python?

Question

I will start by describing the infrastructure I am working within. It contains multiple proxy servers that uses a load balancer to forward user authentications to the appropriate proxy that are directly tied to an active directory. The authentication uses the credentials and source IP that was used to log into the computer the request is coming from. The server caches the IP and credentials for 60 minutes. I am using a test account specifically for this process and is only used on the unit testing server.

I am working on some automation with selenium webdriver on a remote server using a docker container. I am using python as the scripting language. I am trying to run tests on both internal and external webpages/applications. I was able to get a basic test on an internal website with the following script:

Note: 10.1.54.118 is the server hosting the docker container with the selenium web driver

from selenium import webdriver

from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

browser = webdriver.Remote(command_executor='http://10.1.54.118:4444/wd/hub', desired_capabilities=DesiredCapabilities.CHROME)
browser.get("http://10.0.0.2")

print (browser.find_element_by_tag_name('body').text)

bodyText = browser.find_element_by_tag_name('body').text

print (bodyText)
    
if 'Hello' in bodyText:
    print ('Found hello in body')
else:
    print ('Hello not found in body')

browser.quit()

The script is able to access the internal webpage and print all the text on it.

However, I am experiencing problems trying to run test scripts against external websites.

I have tried the following articles and tutorials and it doesn't seem to work for me.

The articles and tutorials I have tried:

• https://www.seleniumhq.org/docs/04_webdriver_advanced.jsp
• Pass driver ChromeOptions and DesiredCapabilities?
• https://www.programcreek.com/python/example/100023/selenium.webdriver.Remote
• https://github.com/webdriverio/webdriverio/issues/324
• https://www.programcreek.com/python/example/96010/selenium.webdriver.common.desired_capabilities.DesiredCapabilities.CHROME
• Running Selenium Webdriver with a proxy in Python
• how do i set proxy for chrome in python webdriver
• https://docs.proxymesh.com/article/4-python-proxy-configuration

I have tried creating 4 versions of a script to access an external site ie google.com and simply print the text off of it. Every script returns a time out error. I apologize for posting a lot of code but maybe the community is able to see where I am going wrong with the coding aspect.

Code 1:

from selenium import webdriver

from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

PROXY = "10.32.51.169:3128" # IP:PORT or HOST:PORT
desired_capabilities = webdriver.DesiredCapabilities.CHROME.copy()

desired_capabilities['proxy'] = {
    "httpProxy":PROXY,
    "ftpProxy":PROXY,
    "sslProxy":PROXY,
    "socksUsername":"myusername",
    "socksPassword":"mypassword",
    "noProxy":None,
    "proxyType":"MANUAL",
    "class":"org.openqa.selenium.Proxy",
    "autodetect":False
    }
browser = webdriver.Remote('http://10.1.54.118:4444/wd/hub', desired_capabilities)
browser.get("https://www.google.com/")

print (browser.find_element_by_tag_name('body').text)

bodyText = browser.find_element_by_tag_name('body').text

print (bodyText)
   
if 'Hello' in bodyText:
    print ('Found hello in body')
else:
    print ('Hello not found in body')

browser.quit()

Is my code incorrect in any way? Am I able to pass configuration parameters to the docker chrome selenium webdriver or do I need to build the docker container with the proxy settings preconfigured before building it? I look forward to your replies and any help that can point me in the right direction.

Answer 1

A little late on this one, but a couple ideas + improvements:

1. Remove the user/pass from the socks proxy config and add them to your Proxy connection uri.
2. Use the selenium Proxy object to help abstract some of the other bits of the proxy capability.
3. Add the scheme to the proxy connection string.
4. Use a try/finally block to make sure the browser quits despite any failures

Note... I'm using Python3, selenium version 3.141.0, and I'm leaving out the FTP config for brevity/simplicity:

from selenium import webdriver

from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.common.proxy import Proxy

# Note the addition of the scheme (http) and the user/pass into the connection string.    
PROXY = 'http://myusername:mypassword@10.32.51.169:3128'

# Use the selenium Proxy object to add proxy capabilities
proxy_config = {'httpProxy': PROXY, 'sslProxy': PROXY}
proxy_object = Proxy(raw=proxy_config)
capabilities = DesiredCapabilities.CHROME.copy()
proxy_object.add_to_capabilities(capabilities)

browser = webdriver.Remote('http://10.1.54.118:4444/wd/hub', desired_capabilities=capabilities)

# Use try/finally so the browser quits even if there is an exception
try:
    browser.get("https://www.google.com/")

    print(browser.find_element_by_tag_name('body').text)

    bodyText = browser.find_element_by_tag_name('body').text

    print(bodyText)

    if 'Hello' in bodyText:
        print('Found hello in body')
    else:
        print('Hello not found in body')
finally:
    browser.quit()