Rack Timeout on large JSON payload Rails, Heroku, Unicorn

Question

I am working on a Rails(4.2) app that has been experiencing occasional timeouts when it receives large JSON payloads. The rack-timeout gem indicates that the app is timing out on the following code block, in Unicorn (the line beginning with @socket is responsible).

/gems/unicorn-4.8.3/lib/unicorn/stream_input.rb:51:in kgio_wait_readable

     rv.replace(@rbuf.slice!(0, @rbuf.size))
         until to_read == 0 || eof? || (rv.size > 0 && @chunked)
           @socket.kgio_read(to_read, @buf) or eof!
           filter_body(@rbuf, @buf)
           rv << @rbuf

This is in production only and the last time this occurred the payload was 42kb of json. I have had no success reproducing this locally, even while running unicorn and running the same payload. Increasing the amount and size of dynos on heroku has no effect either.

Any help is appreciated here, Thanks.

Answer 1

The problem is your request are taking more than 30 seconds and herokur's router is terminating the connection .

I think a good solution for you is paginating your json response, you could follow github's documentation https://developer.github.com/v3/#pagination .