Codeship breaks on rails-devise-roles tests

Question

The rails-devise-roles app's Selenium tests for user-delete-spec.rb and user-edit-spec.rb seems to break when running in Codeship's CI. The rspec works fine on my local machine, but breaks in Codeship.

The rspec output I get is as follows:

Failures:

1) User delete user can delete own account
Failure/Error: visit edit_user_registration_path(user)

Selenium::WebDriver::Error::ServerError:
  status code 500
# /home/rof/cache/bundler/ruby/2.3.0/gems/selenium-webdriver-3.4.0/lib/selenium/webdriver/remote/response.rb:72:in `assert_ok'
# /home/rof/cache/bundler/ruby/2.3.0/gems/selenium-webdriver-3.4.0/lib/selenium/webdriver/remote/response.rb:32:in `initialize'
# /home/rof/cache/bundler/ruby/2.3.0/gems/selenium-webdriver-3.4.0/lib/selenium/webdriver/remote/http/common.rb:83:in `new'
# /home/rof/cache/bundler/ruby/2.3.0/gems/selenium-webdriver-3.4.0/lib/selenium/webdriver/remote/http/common.rb:83:in `create_response'
# /home/rof/cache/bundler/ruby/2.3.0/gems/selenium-webdriver-3.4.0/lib/selenium/webdriver/remote/http/default.rb:107:in `request'
# /home/rof/cache/bundler/ruby/2.3.0/gems/selenium-webdriver-3.4.0/lib/selenium/webdriver/remote/http/common.rb:61:in `call'
# /home/rof/cache/bundler/ruby/2.3.0/gems/selenium-webdriver-3.4.0/lib/selenium/webdriver/remote/w3c_bridge.rb:640:in `execute'
# /home/rof/cache/bundler/ruby/2.3.0/gems/selenium-webdriver-3.4.0/lib/selenium/webdriver/remote/w3c_bridge.rb:114:in `create_session'
# /home/rof/cache/bundler/ruby/2.3.0/gems/selenium-webdriver-3.4.0/lib/selenium/webdriver/remote/w3c_bridge.rb:76:in `initialize'
# /home/rof/cache/bundler/ruby/2.3.0/gems/selenium-webdriver-3.4.0/lib/selenium/webdriver/firefox/w3c_bridge.rb:45:in `initialize'
# /home/rof/cache/bundler/ruby/2.3.0/gems/selenium-webdriver-3.4.0/lib/selenium/webdriver/common/driver.rb:52:in `new'
# /home/rof/cache/bundler/ruby/2.3.0/gems/selenium-webdriver-3.4.0/lib/selenium/webdriver/common/driver.rb:52:in `for'
# /home/rof/cache/bundler/ruby/2.3.0/gems/selenium-webdriver-3.4.0/lib/selenium/webdriver.rb:88:in `for'
# /home/rof/cache/bundler/ruby/2.3.0/gems/capybara-2.14.0/lib/capybara/selenium/driver.rb:22:in `browser'
# /home/rof/cache/bundler/ruby/2.3.0/gems/capybara-2.14.0/lib/capybara/selenium/driver.rb:62:in `visit'
# /home/rof/cache/bundler/ruby/2.3.0/gems/capybara-2.14.0/lib/capybara/session.rb:269:in `visit'
# /home/rof/cache/bundler/ruby/2.3.0/gems/capybara-2.14.0/lib/capybara/dsl.rb:50:in `block (2 levels) in <module:DSL>'
# ./spec/features/users/user_delete_spec.rb:21:in `block (2 levels) in <top (required)>'

2) User edit user changes email address
Failure/Error: visit edit_user_registration_path(user)

Selenium::WebDriver::Error::ServerError:
  status code 500
# [removed stack trace]

3) User edit user cannot cannot edit another user's profile
Failure/Error: visit edit_user_registration_path(other)

Selenium::WebDriver::Error::ServerError:
  status code 500
# [removed stack trace]

Finished in 23.21 seconds (files took 2.99 seconds to load)
31 examples, 3 failures

Failed examples:

rspec ./spec/features/users/user_delete_spec.rb:18 # User delete user can delete own account
rspec ./spec/features/users/user_edit_spec.rb:18 # User edit user changes email address
rspec ./spec/features/users/user_edit_spec.rb:45 # User edit user cannot cannot edit another user's profile

This is an example of one of the failing scenarios:

scenario 'user can delete own account' do
    # skip 'skip a slow test' [removed from the original file]
    user = FactoryGirl.create(:user)
    login_as(user, :scope => :user)
    visit edit_user_registration_path(user)
    click_button 'Cancel my account'
    page.driver.browser.switch_to.alert.accept
    expect(page).to have_content I18n.t 'devise.registrations.destroyed'
end

What's going wrong here, and how can I modify this so the test passes? (To clarify, there is no error 500 on my local deployment on any of these pages, nor on the Heroku live deployment of my app which is based off rails-devise-roles.)

Answer 1

Reached out to Codeship support, who were very helpful, and told me that this was an issue with both Firefox and geckodriver for Firefox needing to be updated on the Codeship VM. If anyone else also has this issue, the code that makes it work is below:

export FIREFOX_VERSION=53.0.3
curl -sSL https://raw.githubusercontent.com/codeship/scripts/master/packages/firefox.sh | bash -s
export GECKODRIVER_VERSION=0.17.0
source /dev/stdin <<< "$(curl -sSL https://raw.githubusercontent.com/codeship/scripts/master/languages/rust.sh)"
curl -sSL https://raw.githubusercontent.com/codeship/scripts/master/packages/geckodriver.sh | bash -s

Huge props to Joe Siewert from the Codeship support team for getting this fix out to me!