Error when uploading large file with Paperclip and AWS

Question

My rails app on heroku allows a user to upload a photo to be stored on amazon web services S3 using paperclip.

When the photo's size is above 1.5MB the app seems to time out. What is the best way to solve this problem?

Here is the stack trace if that helps?

E, [2013-10-07T14:15:57.018396 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.12/lib/action_dispatch/middleware/static.rb:62:in `call'
2013-10-07T14:15:57.018974+00:00 app[web.2]: E, [2013-10-07T14:15:57.018430 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:136:in `forward'
2013-10-07T14:15:57.018974+00:00 app[web.2]: E, [2013-10-07T14:15:57.018772 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:143:in `pass'
2013-10-07T14:15:57.018974+00:00 app[web.2]: E, [2013-10-07T14:15:57.018946 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:71:in `call!'
2013-10-07T14:15:57.019152+00:00 app[web.2]: E, [2013-10-07T14:15:57.018983 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:51:in `call'
2013-10-07T14:15:57.019152+00:00 app[web.2]: E, [2013-10-07T14:15:57.019100 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/airbrake-3.1.12/lib/airbrake/user_informer.rb:16:in `_call'
2013-10-07T14:15:57.019279+00:00 app[web.2]: E, [2013-10-07T14:15:57.019147 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/airbrake-3.1.12/lib/airbrake/user_informer.rb:12:in `call'
2013-10-07T14:15:57.019279+00:00 app[web.2]: E, [2013-10-07T14:15:57.019183 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/rack-timeout-0.0.4/lib/rack/timeout.rb:16:in `block in call'
2013-10-07T14:15:57.019279+00:00 app[web.2]: E, [2013-10-07T14:15:57.019217 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/rack-timeout-0.0.4/lib/rack/timeout.rb:16:in `call'
2013-10-07T14:15:57.019279+00:00 app[web.2]: E, [2013-10-07T14:15:57.019252 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/railties-3.2.12/lib/rails/engine.rb:479:in `call'
2013-10-07T14:15:57.019469+00:00 app[web.2]: E, [2013-10-07T14:15:57.019326 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/railties-3.2.12/lib/rails/application.rb:223:in `call'
2013-10-07T14:15:57.019469+00:00 app[web.2]: E, [2013-10-07T14:15:57.019369 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/railties-3.2.12/lib/rails/railtie/configurable.rb:30:in `method_missing'
2013-10-07T14:15:57.019648+00:00 app[web.2]: E, [2013-10-07T14:15:57.019502 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/unicorn-4.6.3/lib/unicorn/http_server.rb:552:in `process_client'
2013-10-07T14:15:57.019710+00:00 app[web.2]: E, [2013-10-07T14:15:57.019594 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/unicorn-4.6.3/lib/unicorn/http_server.rb:632:in `worker_loop'
2013-10-07T14:15:57.020069+00:00 app[web.2]: E, [2013-10-07T14:15:57.019940 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/unicorn-4.6.3/lib/unicorn/http_server.rb:500:in `spawn_missing_workers'
2013-10-07T14:15:57.020154+00:00 app[web.2]: E, [2013-10-07T14:15:57.020083 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/unicorn-4.6.3/lib/unicorn/http_server.rb:142:in `start'
2013-10-07T14:15:57.020271+00:00 app[web.2]: E, [2013-10-07T14:15:57.020153 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/unicorn-4.6.3/bin/unicorn:126:in `<top (required)>'
2013-10-07T14:15:57.020372+00:00 app[web.2]: E, [2013-10-07T14:15:57.020263 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/bin/unicorn:23:in `load'
2013-10-07T14:15:57.020484+00:00 app[web.2]: E, [2013-10-07T14:15:57.020370 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/bin/unicorn:23:in `<main>'
2013-10-07T14:15:57.022713+00:00 heroku[router]: at=info method=POST path=/shares host=mentionengine.com fwd="105.228.65.98" dyno=web.2 connect=2ms service=15098ms status=500 bytes=0
2013-10-07T14:15:57.017996+00:00 app[web.2]: E, [2013-10-07T14:15:57.017963 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/multipart/parser.rb:116:in `get_current_head_and_filename_and_content_type_and_name_and_body'
2013-10-07T14:15:57.018284+00:00 app[web.2]: E, [2013-10-07T14:15:57.018362 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/lock.rb:15:in `call'
2013-10-07T14:15:57.018284+00:00 app[web.2]: E, [2013-10-07T14:15:57.018294 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/runtime.rb:17:in `call'
2013-10-07T14:15:57.017996+00:00 app[web.2]: E, [2013-10-07T14:15:57.017931 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/unicorn-4.6.3/lib/unicorn/tee_input.rb:84:in `read'
2013-10-07T14:15:57.017884+00:00 app[w
eb.2]: E, [2013-10-07T14:15:57.017831 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/unicorn-4.6.3/lib/unicorn/stream_input.rb:51:in `kgio_wait_readable'
2013-10-07T14:15:57.018284+00:00 app[web.2]: E, [2013-10-07T14:15:57.018262 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/methodoverride.rb:14:in `call'
2013-10-07T14:15:57.018170+00:00 app[web.2]: E, [2013-10-07T14:15:57.018154 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/request.rb:336:in `parse_multipart'
2013-10-07T14:15:57.019942+00:00 app[web.2]: E, [2013-10-07T14:15:57.019819 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.6.6.147/lib/new_relic/agent/instrumentation/unicorn_instrumentation.rb:22:in `block (4 levels) in <top (required)>'
2013-10-07T14:15:57.018170+00:00 app[web.2]: E, [2013-10-07T14:15:57.018074 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/multipart/parser.rb:17:in `parse'
2013-10-07T14:15:57.018170+00:00 app[web.2]: E, [2013-10-07T14:15:57.018004 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/multipart/parser.rb:19:in `block in parse'
2013-10-07T14:15:57.018284+00:00 app[web.2]: E, [2013-10-07T14:15:57.018327 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.12/lib/active_support/cache/strategy/local_cache.rb:72:in `call'
2013-10-07T14:15:57.018170+00:00 app[web.2]: E, [2013-10-07T14:15:57.018038 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/multipart/parser.rb:17:in `loop'
2013-10-07T14:15:57.017884+00:00 app[web.2]: E, [2013-10-07T14:15:57.017899 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/unicorn-4.6.3/lib/unicorn/stream_input.rb:51:in `read'
2013-10-07T14:15:57.017884+00:00 app[web.2]: E, [2013-10-07T14:15:57.017867 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/unicorn-4.6.3/lib/unicorn/stream_input.rb:51:in `kgio_read'
2013-10-07T14:15:57.018284+00:00 app[web.2]: E, [2013-10-07T14:15:57.018231 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/methodoverride.rb:26:in `method_override'
2013-10-07T14:15:57.018170+00:00 app[web.2]: E, [2013-10-07T14:15:57.018191 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/request.rb:201:in `POST'
2013-10-07T14:15:57.019827+00:00 app[web.2]: E, [2013-10-07T14:15:57.019712 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/newrelic_rpm-3.6.6.147/lib/new_relic/agent/instrumentation/unicorn_instrumentation.rb:22:in `call'
2013-10-07T14:15:57.017884+00:00 app[web.2]: E, [2013-10-07T14:15:57.017704 #5] ERROR -- : app error: execution expired (Timeout::Error)
2013-10-07T14:15:57.018170+00:00 app[web.2]: E, [2013-10-07T14:15:57.018114 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/multipart.rb:25:in `parse_multipart'
2013-10-07T14:15:57.018974+00:00 app[web.2]: E, [2013-10-07T14:15:57.018809 #5] ERROR -- : /app/vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:155:in `invalidate'

Answer 1

Heroku has a 30 second timeout on all request processing through the Router Mesh to your dyno.

It looks like your upload is large enough it needs more time to complete, you can't run a request of that size/length through Heroku. Docs are here, explaining their policy: https://devcenter.heroku.com/articles/request-timeout

One way to work-around this limitation is to create an Amazon S3 bucket with public-write allowed that your clients can upload to directly, with JS or a rich client.

Here's a separate question explaining how to us JS to upload directly to S3: Uploading Image to Amazon s3 with HTML, javascript & jQuery with Ajax Request (No PHP)