Rails HTTP request raises `EOFError: end of file reached` after some time

Question

I have two Rails apps that are communicating through HTTP.

The app that is supposed to receive requests takes a long time to respond. I expect that, so using my preferred request method (provided by the httparty gem) I increase the timeout and submit the request:

HTTParty.get("https://example.com", :timeout => 99999)

After approx.2 minutes the error EOFError: end of file reached is raised by the requesting app, with the following stack trace:

from .rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/protocol.rb:153:in `read_nonblock'
from .rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/protocol.rb:153:in `rbuf_fill'
from .rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/protocol.rb:134:in `readuntil'
from .rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/protocol.rb:144:in `readline'
from .rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http/response.rb:39:in `read_status_line'
from .rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http/response.rb:28:in `read_new'
from .rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http.rb:1414:in `block in transport_request'
from .rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http.rb:1411:in `catch'
from .rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http.rb:1411:in `transport_request'
from .rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http.rb:1384:in `request'
from .rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http.rb:1377:in `block in request'
from .rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http.rb:853:in `start'
from .rvm/rubies/ruby-2.2.2/lib/ruby/2.2.0/net/http.rb:1375:in `request'
from .rvm/gems/ruby-2.2.2/gems/httparty-0.13.7/lib/httparty/request.rb:117:in `perform'
from .rvm/gems/ruby-2.2.2/gems/httparty-0.13.7/lib/httparty.rb:545:in `perform_request'
from .rvm/gems/ruby-2.2.2/gems/httparty-0.13.7/lib/httparty.rb:476:in `get'
from .rvm/gems/ruby-2.2.2/gems/httparty-0.13.7/lib/httparty.rb:583:in `get'
from (irb):14
from .rvm/gems/ruby-2.2.2/gems/railties-4.2.6/lib/rails/commands/console.rb:110:in `start'
from .rvm/gems/ruby-2.2.2/gems/railties-4.2.6/lib/rails/commands/console.rb:9:in `start'
from .rvm/gems/ruby-2.2.2/gems/railties-4.2.6/lib/rails/commands/commands_tasks.rb:68:in `console'
from .rvm/gems/ruby-2.2.2/gems/railties-4.2.6/lib/rails/commands/commands_tasks.rb:39:in `run_command!'
from .rvm/gems/ruby-2.2.2/gems/railties-4.2.6/lib/rails/commands.rb:17:in `<top (required)>'
from bin/rails:4:in `require'

But in the log of my receiving app I can see that the request is still being processed. So the requesing app must be raising the exception based on something else that a response.

Any idea how to avoid this?

Answer 1

1. Found the solution in an issue on github: https://github.com/jnunemaker/httparty/issues/594

The essence is that some servers could block requests that contain ruby in their user agent/name (which happens because "ruby" is the user agent used by httparty).

Try by specifying the user agent as such (note rby without the u ):

response = HTTParty.get(my_url, headers: {"User-Agent" => "Why would you block rby agent? Come on"})

2. It could also be that the server has an error and really sends over incomplete data. Test it out with a server you control and can check that the response is properly formatted (proper json, not missing backticks).