在Rails 4中使用capybara-webkit gem时，为什么会偶尔断开连接？

Question

我使用capybara-webkit gem从Rails应用程序的某些页面中抓取数据。 我注意到，似乎是“随机” /“零星”的应用程序将崩溃，并出现以下错误：

Capybara::Webkit::ConnectionError: /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-webkit-1.11.1/bin/webkit_server failed to start.
    from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-webkit-1.11.1/lib/capybara/webkit/server.rb:56:in `parse_port'
    from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-webkit-1.11.1/lib/capybara/webkit/server.rb:42:in `discover_port'
    from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-webkit-1.11.1/lib/capybara/webkit/server.rb:26:in `start'
    from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-webkit-1.11.1/lib/capybara/webkit/connection.rb:67:in `start_server'
    from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-webkit-1.11.1/lib/capybara/webkit/connection.rb:17:in `initialize'
    from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-webkit-1.11.1/lib/capybara/webkit/driver.rb:16:in `new'
    from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-webkit-1.11.1/lib/capybara/webkit/driver.rb:16:in `initialize'
    from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-webkit-1.11.1/lib/capybara/webkit.rb:15:in `new'
    from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-webkit-1.11.1/lib/capybara/webkit.rb:15:in `block in <top (required)>'
    from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-2.7.1/lib/capybara/session.rb:85:in `driver'
    from /home/daveomcd/.rvm/gems/ruby-2.3.1/gems/capybara-2.7.1/lib/capybara/session.rb:233:in `visit'

即使之前已经连接并多次访问网站，它也会发生。 这是我目前正在使用的代码片段...

if site.url.present?

  begin
    # Visit the URL
    session = Capybara::Session.new(:webkit)
    session.visit(site.url)  # here is where the error occurs...
    document = Nokogiri::HTML.parse(session.body)

    # Load configuration options for Development Group
    roster_table_selector = site.development_group.table_selector
    header_row_selector   = site.development_group.table_header_selector
    row_selector   = site.development_group.table_row_selector
    row_offset     = site.development_group.table_row_selector_offset
    header_format_type    = site.config_header_format_type

    # Get the Table and Header Row for processing
    roster_table        = document.css(roster_table_selector)
    header_row          = roster_table.css(header_row_selector)
    header_hash         = retrieve_headers(header_row, header_format_type)

    my_object = process_rows(roster_table, header_hash, site, row_selector, row_offset)

  rescue ::Capybara::Webkit::ConnectionError => e
    raise e

  rescue OpenURI::HTTPError => e
    if e.message == '404 Not Found'
      raise "404 Page not found..."
    else
      raise e
    end
  end
end

我什至以为也许我不知道为什么一定要发生这种情况-但只要恢复就可以。 因此，我打算在急救块中进行“重试”以解决错误，但似乎服务器已关闭-因此重试时我得到的结果相同。 也许有人知道我可以检查服务器是否关闭并重新启动然后执行重试的方法？ 谢谢您的帮助！

Answer 1

因此，在进一步调查之后，似乎为每次循环迭代生成了一个新的Capybara::Session 。 我将其移出循环之外，还添加了Capybara.reset_sessions! 在我的循环结束时。 不确定是否可以解决任何问题-但问题似乎已经解决。 下一个小时左右，我将对其进行监控。 下面是我的ActiveJob代码的示例...

class ScrapeJob < ActiveJob::Base
  queue_as :default
  include Capybara::DSL

  def perform(*args)

    session = Capybara::Session.new(:webkit)

    Site.where(config_enabled: 1).order(:code).each do |site|
      process_roster(site, session)
      Capybara.reset_sessions!
    end

  end

  def process_roster(site, session)

    if site.roster_url.present?

      begin
        # Visit the Roster URL 
        session.visit(site.roster_url)
        document = Nokogiri::HTML.parse(session.body)

        # processing code...

        # pass the session that was created as the final parameter..
        my_object = process_rows( ..., session)

      rescue ::Capybara::Webkit::ConnectionError => e
        raise e

      rescue OpenURI::HTTPError => e
        if e.message == '404 Not Found'
          raise "404 Page not found..."
        else
          raise e
        end
      end
    end
  end
end