Getting Error in Selenium/Python - chromedriver executable needs to be in PATH

Question

First of all, i know that this question has been asked quite a lot here.Just making it clear that i have read most accepted answers and followed all steps properly, still getting this error as mentioned in question. Here's my script down below along with all other information

[ I am using selenium with chromedriver trying to use it with chromium browser in headless mode running ubuntu 20 ]

cat test.py

from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
options.binary_location = '/usr/bin/chromium-browser'

driver = webdriver.Chrome(options=options, executable_path='/usr/local/bin/chromedriver')
driver.get("https://www.google.com")
print(driver.title)
print ("Headless Chrome Initialized")
driver.quit()

python3 test.py

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/common/service.py", line 72, in start                                           self.process = subprocess.Popen(cmd, env=self.env,
  File "/usr/lib/python3.8/subprocess.py", line 854, in __init__           self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.8/subprocess.py", line 1702, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)       FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/bin/chromedriver'
                                                                       During handling of the above exception, another exception occurred:    
Traceback (most recent call last):
  File "test.py", line 9, in <module>
    driver = webdriver.Chrome(options=options, executable_path='/usr/local/bin/chromedriver')
  File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
    self.service.start()
  File "/usr/local/lib/python3.8/dist-packages/selenium/webdriver/common/service.py", line 81, in start
    raise WebDriverException(
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home

What steps i followed

sudo apt install chromium-browser
python3 -m pip install selenium
wget https://chromedriver.storage.googleapis.com/83.0.4103.39/chromedriver_linux64.zip
unzip chromedriver*
chmod +x chromedriver
sudo mv chromedriver /usr/local/bin/
sudo chown root:root /usr/local/bin/chromedriver
sudo chmod 0755 /usr/local/bin/chromedriver

Verifying Everything together

╭─[localhost] as xd003 in ~
╰─➤ lsb_release -a | grep Description && \
apt -qq list python3 && \
apt -qq list chromium-browser && \
which chromedriver && \
which chromium-browser && \
echo $PATH

No LSB modules are available.
Description:    Ubuntu 20.04 LTS
python3/focal,now 3.8.2-0ubuntu2 arm64 [installed]
chromium-browser/focal-updates,now 81.0.4044.129-0ubuntu0.20.04.1 arm64 [installed]
/usr/local/bin/chromedriver
/usr/bin/chromium-browser
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games

Everything looks good me, can someone please suggest what am i doing wrong?

The Ubuntu20 i am running locally is a Ubuntu Proot which i am using on my android device

Answer 1

After days of figuring this out, i finally was able to get this error fixed.

So as i had mentioned i am running Prooted Ubuntu 20 on Android Device. Now here the linux kernel architecture being aarch64 seems not supported by the official chrome webdriver - https://chromedriver.chromium.org

To fix this, just download a chromedriver compiled for arm64 variant.You can always get the latest one from here - https://github.com/electron/electron/releases

In short, everything else was perfect,just needed to replace the wget link by - https://github.com/electron/electron/releases/download/v8.3.4/chromedriver-v8.3.4-linux-arm64.zip

PS / Maybe if the error was something like - linux kernel architecture not supported, it would have been lot easier to fix it but nevermind.

Answer 2

1. chromedriver I always pass to constructor
2. PATH error occurs if it cannot be found (see example below where I deliberately pass incorrect param)
3. this driver does not work with chromium, only chrome. Hence reason to keep drivers separate from /usr/bin dir

import time, os, datetime, json, requests, re, subprocess
from distutils.version import StrictVersion
from pathlib import Path
import xmlplain
import selenium
from selenium import webdriver

def get_release(v):
    r = re.compile(f'^(83\.[0-9]+\.[0-9]+).*linux.*$')
    maxrel = "0.0.0"
    pos = 0
    for i, rel in enumerate(v):
        if r.match(rel) is not None and StrictVersion(r.split(rel)[1]) > StrictVersion(maxrel): 
            maxrel = r.split(rel)[1]
            pos = i

    return v[pos]

def cmd(cmd:str, assertfail=True) :
    up = subprocess.Popen(cmd, shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, close_fds=True)
    str = [l.rstrip().decode() for l in up.stdout]
    exitcode = up.wait()
    if assertfail: assert exitcode == 0, f"[{exitcode}] {cmd} {str}"
    return exitcode,  str


if __name__ == '__main__':
    # os.environ["DISPLAY"] = ":0.0"

    # find latest driver available
    resp = requests.get("https://chromedriver.storage.googleapis.com")
    raw = xmlplain.xml_to_obj(resp.content, strip_space=True, fold_dict=False)
    v = get_release([e["Contents"][0]["Key"] for e in raw["ListBucketResult"] if "Contents" in e.keys()])
    # download if required
    if not Path.cwd().joinpath(v.split("/")[0]).is_dir():
        cmd(f"mkdir {v.split('/')[0]}")
        cmd(f"wget https://chromedriver.storage.googleapis.com/{v} -O {v}")
        cmd(f"unzip {v} -d {v.split('/')[0]}/")
    # hit a webpage
    try:
        driver = webdriver.Chrome(f"unknown/chromedriver")  # Optional argument, if not specified will search path.
    except selenium.common.exceptions.WebDriverException as e:
        print(e)
    options = webdriver.ChromeOptions()
    options.add_argument('--no-sandbox')
    options.add_argument('--headless')
    options.add_argument('--disable-dev-shm-usage')
    options.binary_location = '/usr/bin/chromium-browser'
    options.binary_location = '/usr/bin/google-chrome'
    driver = webdriver.Chrome(f"{v.split('/')[0]}/chromedriver", options=options)  # Optional argument, if not specified will search path.
    driver.get('https://www.google.com')
    print(driver.title)
    print ("Headless Chrome Initialized")
    driver.quit()

output:

Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home

Google
Headless Chrome Initialized