Capybara, Selenium, headless Chrome, Ubuntu Net::ReadTimeout

Question

I saw a very similar question to mine. Everything works fine on OS X, but throws errors on ubuntu 14.04. Retracing that post, I am getting the same error. The original poster gave up and used Poltergeist / PhantomJS instead. I'd like to make this work with Chrome now that PhantomJS is no longer being maintained.

Net::ReadTimeout: Net::ReadTimeout and Selenium::WebDriver::Error::UnknownError: unknown error: Chrome failed to start on Rails 5.1.beta System Test

Here are my install steps on Ubuntu:

apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 40976EAF437D05B5
wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - 
cat <<EOF > /etc/apt/sources.list.d/google-chrome.list
deb http://dl.google.com/linux/chrome/deb/ stable main
EOF
apt-get update
apt-get install --force-yes -y google-chrome-stable
cd /root/ && curl -O "http://chromedriver.storage.googleapis.com/2.32/chromedriver_linux64.zip"
cd /root/ && unzip chromedriver_linux64.zip && cp chromedriver /usr/bin

Taking one step at a time, I confirmed google-chrome, chromedriver installed correctly and work fine. Selenium driver also works fine.

link = ENV['LINK'] || "https://www.amazon.com"
#https://stackoverflow.com/questions/44424200/how-do-i-use-selenium-webdriver-on-headless-chrome
Selenium::WebDriver::Chrome.driver_path="/usr/bin/chromedriver" if RUBY_PLATFORM.include? "linux"
options = %w[--headless --disable-gpu]
options +=  %w[--binary='/usr/bin/google-chrome'] if RUBY_PLATFORM.include? "linux"
driver = Selenium::WebDriver.for :chrome, switches: options
driver.navigate.to "#{link}"
driver.save_screenshot("./screen.png")
driver.quit

The Capybara test times out while visiting the url.

require 'capybara'
include Capybara::DSL

link = ENV['LINK'] || "https://www.amazon.com"
options = %w[--headless --disable-gpu]
options +=  %w[--binary='/usr/bin/google-chrome'] if RUBY_PLATFORM.include? "linux"
Selenium::WebDriver::Chrome.driver_path="/usr/bin/chromedriver" if RUBY_PLATFORM.include? "linux"
Capybara.register_driver(:headless_chrome) do |app|
    capabilities = Selenium::WebDriver::Remote::Capabilities.chrome( chromeOptions: { args: options } )
    Capybara::Selenium::Driver.new(app, browser: :chrome, desired_capabilities: capabilities )
end
Capybara.javascript_driver = :headless_chrome
session = Capybara::Session.new(:headless_chrome)
session.visit "#{link}"

The error is the following msg:

session.visit "#{link}"

Net::ReadTimeout: Net::ReadTimeout
    from /usr/lib/ruby/2.3.0/net/protocol.rb:158:in `rbuf_fill'
    from /usr/lib/ruby/2.3.0/net/protocol.rb:136:in `readuntil'
    from /usr/lib/ruby/2.3.0/net/protocol.rb:146:in `readline'
    from /usr/lib/ruby/2.3.0/net/http/response.rb:40:in `read_status_line'
    from /usr/lib/ruby/2.3.0/net/http/response.rb:29:in `read_new'
    from /usr/lib/ruby/2.3.0/net/http.rb:1437:in `block in transport_request'
    from /usr/lib/ruby/2.3.0/net/http.rb:1434:in `catch'
    from /usr/lib/ruby/2.3.0/net/http.rb:1434:in `transport_request'
    from /usr/lib/ruby/2.3.0/net/http.rb:1407:in `request'
    from /usr/lib/ruby/2.3.0/net/http.rb:1400:in `block in request'
    from /usr/lib/ruby/2.3.0/net/http.rb:853:in `start'
    from /usr/lib/ruby/2.3.0/net/http.rb:1398:in `request'
    from /var/www/railsapp/vendor/bundle/ruby/2.3.0/gems/selenium-webdriver-2.53.4/lib/selenium/webdriver/remote/http/default.rb:107:in `response_for'
    from /var/www/railsapp/vendor/bundle/ruby/2.3.0/gems/selenium-webdriver-2.53.4/lib/selenium/webdriver/remote/http/default.rb:58:in `request'
    from /var/www/railsapp/vendor/bundle/ruby/2.3.0/gems/selenium-webdriver-2.53.4/lib/selenium/webdriver/remote/http/common.rb:59:in `call'
    from /var/www/railsapp/vendor/bundle/ruby/2.3.0/gems/selenium-webdriver-2.53.4/lib/selenium/webdriver/remote/bridge.rb:649:in `raw_execute'
... 10 levels...
    from /usr/bin/irb:11:in `<top (required)>'
    from /usr/lib/ruby/gems/2.3.0/gems/bundler-1.13.6/lib/bundler/cli/exec.rb:74:in `load'
    from /usr/lib/ruby/gems/2.3.0/gems/bundler-1.13.6/lib/bundler/cli/exec.rb:74:in `kernel_load'
    from /usr/lib/ruby/gems/2.3.0/gems/bundler-1.13.6/lib/bundler/cli/exec.rb:27:in `run'
    from /usr/lib/ruby/gems/2.3.0/gems/bundler-1.13.6/lib/bundler/cli.rb:332:in `exec'
    from /usr/lib/ruby/gems/2.3.0/gems/bundler-1.13.6/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
    from /usr/lib/ruby/gems/2.3.0/gems/bundler-1.13.6/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
    from /usr/lib/ruby/gems/2.3.0/gems/bundler-1.13.6/lib/bundler/vendor/thor/lib/thor.rb:359:in `dispatch'
    from /usr/lib/ruby/gems/2.3.0/gems/bundler-1.13.6/lib/bundler/cli.rb:20:in `dispatch'
    from /usr/lib/ruby/gems/2.3.0/gems/bundler-1.13.6/lib/bundler/vendor/thor/lib/thor/base.rb:440:in `start'
    from /usr/lib/ruby/gems/2.3.0/gems/bundler-1.13.6/lib/bundler/cli.rb:11:in `start'
    from /usr/lib/ruby/gems/2.3.0/gems/bundler-1.13.6/exe/bundle:34:in `block in <top (required)>'
    from /usr/lib/ruby/gems/2.3.0/gems/bundler-1.13.6/lib/bundler/friendly_errors.rb:100:in `with_friendly_errors'
    from /usr/lib/ruby/gems/2.3.0/gems/bundler-1.13.6/exe/bundle:26:in `<top (required)>'
    from /usr/bin/bundle:23:in `load'
    from /usr/bin/bundle:23:in `<main>'

Answer 1

Headless Chrome and the chromedriver that support it are relatively new. Update your selenium-webdriver to the latest versions (3.5.2 as of this answer) to have support for it. Once you've done that, if you're using the latest Capybara, you can also try just using the Capybara provided registered driver, with

if RUBY_PLATFORM.include? "linux"
  Selenium::WebDriver::Chrome.driver_path = "/usr/bin/chromedriver"
  Selenium::WebDriver::Chrome.path = "/usr/bin/google-chrome"
end
Capybara.javascript_driver = :selenium_chrome_headless

rather than needing to register your own driver.