在 Heroku 上使用 Python selenium 运行 ChromeDriver

Question

So I have a Flask server on Heroku which has been working fine as expected for some time.Now, as per new requirements, I need to add functionality to the Flask server to fetch a page from an external website.Because of reasons best known to me, I am using Selenium along with Chrome web driver to do this.Locally I was able to set this up and it works fine but I am quite unsure as to how to set it up on the Heroku server. I read a bit about buildpacks and found this buildpack for ChromeDriver :

https://elements.heroku.com/buildpacks/jimmynguyc/heroku-buildpack-chromedriver

However, I am not sure how to proceed further.How do I install chromium browser itself and what else is needed to tie it all up ?

Answer 1

I had the same issue and the following steps worked fine for me:

• I added the following buildpacks on heroku: https://github.com/heroku/heroku-buildpack-xvfb-google-chrome (to install chrome, since chromedriver requires it) and https://github.com/heroku/heroku-buildpack-chromedriver .
• I created an environment variable GOOGLE_CHROME_BIN, with the path of chrome on heroku: /app/.apt/usr/bin/google-chrome and an environment variable called CHROMEDRIVER_PATH with the path of chromedriver on heroku: /app/.chromedriver/bin/chromedriver.

• In my python file, I configured chromedriver:

   chrome_options = Options() chrome_options.binary_location = GOOGLE_CHROME_BIN chrome_options.add_argument('--disable-gpu') chrome_options.add_argument('--no-sandbox') driver = webdriver.Chrome(executable_path=CHROMEDRIVER_PATH, chrome_options=chrome_options)

(First, I tried to configure chromedriver with no arguments, but I faced the following error: "Chrome failed to start: crashed". --disable-gpu and --no-sandbox solved the problem for me).

Answer 2

In your Heroku app go to Settings and add the following build packs:

• https://github.com/heroku/heroku-buildpack-chromedriver
• https://github.com/heroku/heroku-buildpack-google-chrome

In addition, in your Python script you have to set a few Chrome options so that your script runs on Heroku without error.

import time

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager

gChromeOptions = webdriver.ChromeOptions()
gChromeOptions.add_argument("window-size=1920x1480")
gChromeOptions.add_argument("disable-dev-shm-usage")
gDriver = webdriver.Chrome(
    chrome_options=gChromeOptions, executable_path=ChromeDriverManager().install()
)
gDriver.get("https://www.python.org/")
time.sleep(3)
gDriver.save_screenshot("my_screenshot.png")
gDriver.close()

Here is a more detailed post that I created in case you are still having problems: https://www.jtrocinski.com/posts/Heroku-Use_Selenium_to_run_Google_Chrome_in_Python.html

Answer 3

For pyppeteer

If anybody is running on Heroku and facing the same error:

Add the buildpack : The url for the buildpack is below :

https://github.com/jontewks/puppeteer-heroku-buildpack

Ensure that you're using --no-sandbox mode

launch({ args: ['--no-sandbox'] })