Selenium crashing even though chromedriver is in the right place (EC2 instance)

Question

I'm running into a problem where Selenium crashes with an error indicating the chrome driver can't be found, but I've verified it's in the correct location and is executable, and have exhausted all the other fixes I've found. I'm working on an EC2 instance, trying to run the following code:

scraper.py

from selenium import webdriver
options = webdriver.ChromeOptions()
options.binary_location = '/usr/bin/google-chrome'
options.add_argument('--no-sandbox')
options.add_argument('--headless')
options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome(chrome_options=options)

I've verified that the /usr/bin/google-chrome folder has the chromedriver executable:

[ec2-user@ip-xxx google-chrome]$ pwd
/usr/bin/google-chrome
[ec2-user@ip-xxx google-chrome]$ ls
chromedriver  chromedriver_linux64.zip

and that I can execute the binary file:

[ec2-user@ip-xxx google-chrome]$ ./chromedriver
Starting ChromeDriver 98.0.4758.102 (273bf7ac8c909cde36982d27f66f3c70846a3718-refs/branch-heads/4758@{#1151}) on port 9515
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.

I've also verified that I'm not running as root user:

[ec2-user@ip-xxx google-chrome]$ whoami
ec2-user

Yet I am still getting the error below when I run scraper.py. Any idea why this might be happening?

[ec2-user@ip_xxx code]$ python3 scraper.py
Using selenium to get cnn.com
scraper.py:256: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
  driver = webdriver.Chrome('google-chrome/chromedriver',chrome_options=options)
scraper.py:256: DeprecationWarning: use options instead of chrome_options
  driver = webdriver.Chrome('google-chrome/chromedriver',chrome_options=options)
Traceback (most recent call last):
  File "scraper.py", line 456, in <module>
    cnn_s = CnnScraper()
  File "scraper.py", line 256, in __init__
    driver = webdriver.Chrome('google-chrome/chromedriver',chrome_options=options)
  File "/home/ec2-user/.local/lib/python3.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
    service_log_path, service, keep_alive)
  File "/home/ec2-user/.local/lib/python3.7/site-packages/selenium/webdriver/chromium/webdriver.py", line 99, in __init__
    options=options)
  File "/home/ec2-user/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 268, in __init__
    self.start_session(capabilities, browser_profile)
  File "/home/ec2-user/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 359, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/home/ec2-user/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 424, in execute
    self.error_handler.check_response(response)
  File "/home/ec2-user/.local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally.
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Stacktrace:
#0 0x5614df24cb33 <unknown>
#1 0x5614ded156d8 <unknown>
#2 0x5614ded3884c <unknown>
#3 0x5614ded33fca <unknown>
#4 0x5614ded6ee0a <unknown>
#5 0x5614ded68f53 <unknown>
#6 0x5614ded3ea0a <unknown>
#7 0x5614ded3fad5 <unknown>
#8 0x5614df27e2fd <unknown>
#9 0x5614df2974bb <unknown>
#10 0x5614df2800d5 <unknown>
#11 0x5614df298145 <unknown>
#12 0x5614df273aaf <unknown>
#13 0x5614df2b4ba8 <unknown>
#14 0x5614df2b4d28 <unknown>
#15 0x5614df2cf48d <unknown>
#16 0x7f51004ec44b <unknown>

Answer 1

from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
options = Options()
driver = webdriver.Chrome(service=Service('/usr/bin/google-chrome/chromedriver.exe'),options=options)

You need to include a Service, Options like so to be in line with the recent Selenium updates.