How to set a custom name for the user-agent using Selenium and Python

Question

I am using selenium + webdriver and trying testing different user agents. I am adding user agent like this for Chrome on Windows for example:

option = Options()
option.add_argument("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36")

Now when I log see login details it says Windows Chrome but when I want to rename it to something else like this:

option.add_argument("user-agent=test-user-agent")

or

option.add_argument("user-agent=Mozilla/5.0 (test-user-agent NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36")

Some websites display it as unknown or browser not supported

Is there a way to "rename" user-agent or create custom one or there is only preset number of them that websites know?

Answer 1

User-Agent

The User-Agent request header is a characteristic string that lets servers and network peers identify the application, operating system, vendor, and/or version of the requesting user agent .

---

Syntax

The common format for web browsers is as follows:

User-Agent: Mozilla/5.0 (<system-information>) <platform> (<platform-details>) <extensions>

---

This usecase

While your first code attempt to add a specific user-agent would work perfect:

• Code Block:

   from selenium import webdriver options = webdriver.ChromeOptions() options.add_argument("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36") driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe') print(driver.execute_script("return navigator.userAgent;"))

• Console Output:

   Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36

But as per your second attempt you can't rename the User-Agent as it violates the prescribed format/syntax.

---

However, you can always change the User-Agent using the execute_cdp_cmd(cmd, cmd_args) as follows:

• Code Block:

   from selenium import webdriver options = webdriver.ChromeOptions() options.add_argument("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36") driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe') print(driver.execute_script("return navigator.userAgent;")) # Setting UserAgent as Chrome/83.0.4103.97 driver.execute_cdp_cmd('Network.setUserAgentOverride', {"userAgent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36'}) print(driver.execute_script("return navigator.userAgent;"))

• Console Output:

   Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36

---

References

You can find a couple of relevant detailed discussions in:

• Way to change Google Chrome user agent in Selenium?
• How to change the User Agent using Selenium and Python
• How to rotate various user agents using selenium python on each request

Answer 2

Warning: FirefoxProfile is DEPRICATED

The normal method of using FirefoxProfile is now depricated and you must use Options instead.

How to change FireFox User Agent using Options

Simply do the same as FirefoxProfile but with Options object instead like so:

from selenium import webdriver
from selenium.webdriver.firefox.options import Options

options = Options()
agent = " Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)"
options.set_preference("general.useragent.override", agent)

driver = webdriver.Firefox(options=options)

Using random User Agent from git

To use a random user agent you can just pull from this git list of useragents at random (requires pip install requests ):

import requests,random
agents = requests.get("https://gist.githubusercontent.com/pzb/b4b6f57144aea7827ae4/raw/cf847b76a142955b1410c8bcef3aabe221a63db1/user-agents.txt").text.split('\n')
agent = random.choice(agents)

Using random user agent from pip package

There is a helpful package called random-user-agent by installing through pip

pip install random-user-agent

Then to get a random agent:

from fake_useragent import UserAgent
agent = UserAgent().random

Full Example

Here is a full example of setting a random user agent for selenium using options:

from fake_useragent import UserAgent
from selenium import webdriver
from selenium.webdriver.firefox.options import Options

options = Options()
options.set_preference("general.useragent.override", UserAgent().random)

driver = webdriver.Firefox(options=options)

Cheers