Error Starting Rails App In Staging - Unicorn

Question

So I just deployed a Rails 4 app to a AWS EC2 instance running Nginx + Unicorn. I think everything is running fine on Unicorn's end because the requests are hitting the Rails app, but I am getting this weird 500 error and can't tell where it is coming from.

Here is what I am getting from my staging.log . I have no idea what is happening, any help would be great.

I, [2014-05-25T05:11:29.460247 #21589]  INFO -- : Started GET "/" for <ip> at 2014-05-25 05:11:29 +0000
I, [2014-05-25T05:11:29.483476 #21589]  INFO -- : Processing by PlaybookController#index as HTML
I, [2014-05-25T05:11:29.486810 #21589]  INFO -- : Completed 500 Internal Server Error in 3ms
F, [2014-05-25T05:11:29.491919 #21589] FATAL -- :
TypeError (no implicit conversion of Array into String):
  activesupport (4.1.1) lib/active_support/key_generator.rb:21:in `pbkdf2_hmac_sha1'
  activesupport (4.1.1) lib/active_support/key_generator.rb:21:in `generate_key'
  activesupport (4.1.1) lib/active_support/key_generator.rb:38:in `generate_key'
  actionpack (4.1.1) lib/action_dispatch/middleware/cookies.rb:509:in `initialize'
  actionpack (4.1.1) lib/action_dispatch/middleware/cookies.rb:160:in `new'
  actionpack (4.1.1) lib/action_dispatch/middleware/cookies.rb:160:in `encrypted'
  actionpack (4.1.1) lib/action_dispatch/middleware/cookies.rb:169:in `signed_or_encrypted'
  actionpack (4.1.1) lib/action_dispatch/middleware/session/cookie_store.rb:119:in `cookie_jar'
  actionpack (4.1.1) lib/action_dispatch/middleware/session/cookie_store.rb:114:in `get_cookie'
  actionpack (4.1.1) lib/action_dispatch/middleware/session/cookie_store.rb:90:in `block in unpacked_cookie_data'
  actionpack (4.1.1) lib/action_dispatch/middleware/session/abstract_store.rb:51:in `stale_session_check!'
  actionpack (4.1.1) lib/action_dispatch/middleware/session/cookie_store.rb:89:in `unpacked_cookie_data'
  actionpack (4.1.1) lib/action_dispatch/middleware/session/cookie_store.rb:83:in `block in extract_session_id'
  actionpack (4.1.1) lib/action_dispatch/middleware/session/abstract_store.rb:51:in `stale_session_check!'
  actionpack (4.1.1) lib/action_dispatch/middleware/session/cookie_store.rb:82:in `extract_session_id'
  actionpack (4.1.1) lib/action_dispatch/request/session.rb:49:in `block in []'
  actionpack (4.1.1) lib/action_dispatch/request/session.rb:48:in `fetch'
  actionpack (4.1.1) lib/action_dispatch/request/session.rb:48:in `[]'
  actionpack (4.1.1) lib/action_dispatch/request/session.rb:70:in `id'
  rack (1.5.2) lib/rack/session/abstract/id.rb:282:in `current_session_id'
  rack (1.5.2) lib/rack/session/abstract/id.rb:288:in `session_exists?'
  actionpack (4.1.1) lib/action_dispatch/request/session.rb:152:in `exists?'
  actionpack (4.1.1) lib/action_dispatch/request/session.rb:172:in `load_for_read!'
  actionpack (4.1.1) lib/action_dispatch/request/session.rb:89:in `[]'
  turbolinks (2.2.2) lib/turbolinks/xhr_headers.rb:35:in `set_xhr_redirected_to'
  activesupport (4.1.1) lib/active_support/callbacks.rb:424:in `block in make_lambda'
  activesupport (4.1.1) lib/active_support/callbacks.rb:160:in `call'
  activesupport (4.1.1) lib/active_support/callbacks.rb:160:in `block in halting'
  activesupport (4.1.1) lib/active_support/callbacks.rb:166:in `call'
  activesupport (4.1.1) lib/active_support/callbacks.rb:166:in `block in halting'
  activesupport (4.1.1) lib/active_support/callbacks.rb:86:in `call'
  activesupport (4.1.1) lib/active_support/callbacks.rb:86:in `run_callbacks'
  actionpack (4.1.1) lib/abstract_controller/callbacks.rb:19:in `process_action'
  actionpack (4.1.1) lib/action_controller/metal/rescue.rb:29:in `process_action'
  actionpack (4.1.1) lib/action_controller/metal/instrumentation.rb:31:in `block in process_action'
  activesupport (4.1.1) lib/active_support/notifications.rb:159:in `block in instrument'
  activesupport (4.1.1) lib/active_support/notifications/instrumenter.rb:20:in `instrument'
  activesupport (4.1.1) lib/active_support/notifications.rb:159:in `instrument'
  actionpack (4.1.1) lib/action_controller/metal/instrumentation.rb:30:in `process_action'
  actionpack (4.1.1) lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
  activerecord (4.1.1) lib/active_record/railties/controller_runtime.rb:18:in `process_action'
 actionpack (4.1.1) lib/abstract_controller/base.rb:136:in `process'
 actionview (4.1.1) lib/action_view/rendering.rb:30:in `process'
 actionpack (4.1.1) lib/action_controller/metal.rb:195:in `dispatch'
 actionpack (4.1.1) lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
 actionpack (4.1.1) lib/action_controller/metal.rb:231:in `block in action'
 actionpack (4.1.1) lib/action_dispatch/routing/route_set.rb:80:in `call'
 actionpack (4.1.1) lib/action_dispatch/routing/route_set.rb:80:in `dispatch'
 actionpack (4.1.1) lib/action_dispatch/routing/route_set.rb:48:in `call' "../../log/staging.log" [readonly] 960L, 73078C                                                                                                                                                                              
 activesupport (4.1.1) lib/active_support/notifications.rb:159:in `instrument'
 actionpack (4.1.1) lib/action_controller/metal/instrumentation.rb:30:in `process_action'
 actionpack (4.1.1) lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
 activerecord (4.1.1) lib/active_record/railties/controller_runtime.rb:18:in `process_action'
 actionpack (4.1.1) lib/abstract_controller/base.rb:136:in `process'
 actionview (4.1.1) lib/action_view/rendering.rb:30:in `process'
 actionpack (4.1.1) lib/action_controller/metal.rb:195:in `dispatch'
 actionpack (4.1.1) lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
 actionpack (4.1.1) lib/action_controller/metal.rb:231:in `block in action'
 actionpack (4.1.1) lib/action_dispatch/routing/route_set.rb:80:in `call'
 actionpack (4.1.1) lib/action_dispatch/routing/route_set.rb:80:in `dispatch'
 actionpack (4.1.1) lib/action_dispatch/routing/route_set.rb:48:in `call'
 actionpack (4.1.1) lib/action_dispatch/journey/router.rb:71:in `block in call'
 actionpack (4.1.1) lib/action_dispatch/journey/router.rb:59:in `each'
 actionpack (4.1.1) lib/action_dispatch/journey/router.rb:59:in `call'
 actionpack (4.1.1) lib/action_dispatch/routing/route_set.rb:676:in `call'
 rack (1.5.2) lib/rack/etag.rb:23:in `call'
 rack (1.5.2) lib/rack/conditionalget.rb:25:in `call'
 rack (1.5.2) lib/rack/head.rb:11:in `call'
 actionpack (4.1.1) lib/action_dispatch/middleware/params_parser.rb:27:in `call'
 actionpack (4.1.1) lib/action_dispatch/middleware/flash.rb:254:in `call'
 rack (1.5.2) lib/rack/session/abstract/id.rb:225:in `context'
 rack (1.5.2) lib/rack/session/abstract/id.rb:220:in `call'
 actionpack (4.1.1) lib/action_dispatch/middleware/cookies.rb:560:in `call'
 activerecord (4.1.1) lib/active_record/query_cache.rb:36:in `call'
 activerecord (4.1.1) lib/active_record/connection_adapters/abstract/connection_pool.rb:621:in `call'
 actionpack (4.1.1) lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
 activesupport (4.1.1) lib/active_support/callbacks.rb:82:in `run_callbacks'
 actionpack (4.1.1) lib/action_dispatch/middleware/callbacks.rb:27:in `call'
 actionpack (4.1.1) lib/action_dispatch/middleware/remote_ip.rb:76:in `call'
 actionpack (4.1.1) lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
 actionpack (4.1.1) lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
 railties (4.1.1) lib/rails/rack/logger.rb:38:in `call_app'
 railties (4.1.1) lib/rails/rack/logger.rb:20:in `block in call'
 activesupport (4.1.1) lib/active_support/tagged_logging.rb:68:in `block in tagged'
 activesupport (4.1.1) lib/active_support/tagged_logging.rb:26:in `tagged'
 activesupport (4.1.1) lib/active_support/tagged_logging.rb:68:in `tagged'
 railties (4.1.1) lib/rails/rack/logger.rb:20:in `call'
 actionpack (4.1.1) lib/action_dispatch/middleware/request_id.rb:21:in `call'
 rack (1.5.2) lib/rack/methodoverride.rb:21:in `call'
 rack (1.5.2) lib/rack/runtime.rb:17:in `call'
 activesupport (4.1.1) lib/active_support/cache/strategy/local_cache_middleware.rb:26:in `call'
 rack (1.5.2) lib/rack/sendfile.rb:112:in `call'
 railties (4.1.1) lib/rails/engine.rb:514:in `call'
 railties (4.1.1) lib/rails/application.rb:144:in `call'
 unicorn (4.8.3) lib/unicorn/http_server.rb:576:in `process_client'
 unicorn (4.8.3) lib/unicorn/http_server.rb:670:in `worker_loop'
 unicorn (4.8.3) lib/unicorn/http_server.rb:525:in `spawn_missing_workers'
 unicorn (4.8.3) lib/unicorn/http_server.rb:140:in `start'
 unicorn (4.8.3) bin/unicorn_rails:209:in `<top (required)>'
 /var/www/playbook/current/bin/unicorn_rails:16:in `load'
 /var/www/playbook/current/bin/unicorn_rails:16:in `<main>'

Answer 1

I am no Ruby expert. But backtrace dumps like this in error logs drive me nuts as well. The key is to look at the top few lines:

I, [2014-05-25T05:11:29.460247 #21589]  INFO -- : Started GET "/" for <ip> at 2014-05-25 05:11:29 +0000
I, [2014-05-25T05:11:29.483476 #21589]  INFO -- : Processing by PlaybookController#index as HTML
I, [2014-05-25T05:11:29.486810 #21589]  INFO -- : Completed 500 Internal Server Error in 3ms
F, [2014-05-25T05:11:29.491919 #21589] FATAL -- :
TypeError (no implicit conversion of Array into String):

The 500 error means there is an internal server error. Meaning there is something wrong in your code—whether it be PHP, Perl or Ruby—that is just killing the server completely. And the key I see here is:

TypeError (no implicit conversion of Array into String):

Somewhere in your Ruby code you are attempting to assign an array as a string. And that is what is killing the whole process. Where? Unsure from looking at the log. But that is the key to this whole thing failing. The rest of the dump is basically a report of all of the items loaded—libraries & such—that support your application.

Answer 2

I had a similar error in the same place: TypeError (no implicit conversion of Fixnum into String) . It was due to the fact that I'd set a purely numeric secret_key_base in config/secrets.yml. Perhaps that property in your yaml is being interpreted as an Array?

The format should be:

staging:
  secret_key_base: 61a6fa38260a0fdfecac5c088aadd91e87

Of course, don't just copy/paste the above, since everyone would know your secret and that would be unsafe.