Rake fails with Capistrano deployment

Question

I am deploying to Ubuntu with Capistrano.

I have God + Resque set up for background process.

App runs fine and Resque workers are going to their queues, but are stuck there. When I looked into logs, I found out that it's Rake failing. Here is the trace:

FIRST THERE IS ABOUT 1000 OF THE BELOW LINE

/home/ubuntu/myapp/releases/20120724024732/config/application.rb:68:in `invoke'
/home/ubuntu/myapp/shared/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/application.rb:116:in `invoke_task'
/home/ubuntu/myapp/shared/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/application.rb:94:in `block (2 levels) in top_level'
/home/ubuntu/myapp/shared/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/application.rb:94:in `each'
/home/ubuntu/myapp/shared/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/application.rb:94:in `block in top_level'
/home/ubuntu/myapp/shared/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/application.rb:133:in `standard_exception_handling'
/home/ubuntu/myapp/shared/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/application.rb:88:in `top_level'
/home/ubuntu/myapp/shared/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/application.rb:66:in `block in run'
/home/ubuntu/myapp/shared/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/application.rb:133:in `standard_exception_handling'
/home/ubuntu/myapp/shared/bundle/ruby/1.9.1/gems/rake-0.9.2.2/lib/rake/application.rb:63:in `run'
/home/ubuntu/myapp/shared/bundle/ruby/1.9.1/gems/rake-0.9.2.2/bin/rake:33:in `<top (required)>'
/home/ubuntu/.rvm/gems/ruby-1.9.3-p125@global/bin/rake:19:in `load'
/home/ubuntu/.rvm/gems/ruby-1.9.3-p125@global/bin/rake:19:in `<main>'
/home/ubuntu/.rvm/gems/ruby-1.9.3-p125/bin/ruby_noexec_wrapper:14:in `eval'
/home/ubuntu/.rvm/gems/ruby-1.9.3-p125/bin/ruby_noexec_wrapper:14:in `<main>'
rake aborted!
Too many open files - rake_tasks_log.log
/home/ubuntu/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open-uri.rb:35:in `initialize'
/home/ubuntu/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open-uri.rb:35:in `open'
/home/ubuntu/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open-uri.rb:35:in `open'
/home/ubuntu/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/logger.rb:592:in `open_logfile'
/home/ubuntu/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/logger.rb:549:in `initialize'
/home/ubuntu/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/logger.rb:314:in `new'
/home/ubuntu/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/logger.rb:314:in `initialize'
/home/ubuntu/myapp/shared/bundle/ruby/1.9.1/gems/activesupport-3.2.6/lib/active_support/core_ext/logger.rb:72:in `initialize'

Here is the function from application.rb line 68 is marked with comment:

module Rake
  class Task
    alias_method :origin_invoke, :invoke if method_defined?(:invoke)
    def invoke(*args)
      logger = Logger.new('rake_tasks_log.log')
      logger.info "#{Time.now} -- #{name} -- #{args.inspect}"
      puts args
      origin_invoke(args) ### LINE 68
    end
  end
end

* *EDIT:: Here is a clue, but don't know what to do with it:: Apparently the *args that are being passed is an empty array...

Here is my deploy.rb

require "bundler/capistrano"


$:.unshift(File.expand_path('./lib', ENV['rvm_path']))
require 'rvm/capistrano'
set :rvm_ruby_string, ENV['GEM_HOME'].gsub(/.*\//,"") 
set :rvm_type, :user


# -*- encoding : utf-8 -*-

set :assets_dependencies, %w(app/assets lib/assets vendor/assets Gemfile.lock config/routes.rb)

namespace :deploy do
  namespace :assets do

    desc <<-DESC
      Run the asset precompilation rake task. You can specify the full path \
      to the rake executable by setting the rake variable. You can also \
      specify additional environment variables to pass to rake via the \
      asset_env variable. The defaults are:

        set :rake,      "rake"
        set :rails_env, "production"
        set :asset_env, "RAILS_GROUPS=assets"
        set :assets_dependencies, fetch(:assets_dependencies) + %w(config/locales/js)
    DESC
    task :precompile, :roles => :web, :except => { :no_release => true } do
      from = source.next_revision(current_revision)
      if capture("cd #{latest_release} && #{source.local.log(from)} #{assets_dependencies.join ' '} | wc -l").to_i > 0
        run %Q{cd #{latest_release} && #{rake} RAILS_ENV=#{rails_env} #{asset_env} assets:precompile}
      else
        logger.info "Skipping asset pre-compilation because there were no asset changes"
      end
    end

  end
end

set :application, "myapp"

set :repository, "git@github.com:..."
set :scm, :git
set :git_shallow_clone, 1
set :branch, "master"

set :location, "ec2-....amazonaws.com"
role :app, location
role :web, location
role :db,  location, :primary => true

set :deploy_via, :remote_cache


set :user, "ubuntu"
set :use_sudo, false
set :deploy_to, "/home/ubuntu/myapp"
ssh_options[:forward_agent] = true
default_run_options[:pty] = true
ssh_options[:keys] = ["#{ENV['HOME']}/...pem"]

set :keep_releases, 1

# setup some Capistrano roles
role :app, location
role :web, location
role :db,  location, :primary => true

set :default_environment, { 
  'PATH' => "/home/ubuntu/.rvm/gems/ruby-1.9.3-p125/bin:/home/ubuntu/.rvm/gems/ruby-1.9.3-p125@global/bin:/home/ubuntu/.rvm/rubies/ruby-1.9.3-p194/bin:/home/ubuntu/.rvm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/lib/pkgconfig:/usr/lib/jvm/java-1.6.0-openjdk-amd64:/usr/lib/jvm/java-6-openjdk-amd64:$PATH",
  'JAVA_HOME' => "/usr/lib/jvm/java-6-openjdk-amd64",
  'PKG_CONFIG_PATH' => '$PKG_CONFIG_PATH:/usr/local/lib/pkgconfig'

}


desc "show path"
task :show_path do
  run "echo $PATH"
end

###PASSENGER:

namespace :passenger do
    desc "Restart Application"
    task :restart do
    run 'sudo /etc/init.d/nginx stop'
        run 'sudo /etc/init.d/nginx start'
    end
end


after :deploy, "passenger:restart"
after :deploy, "deploy:migrate"

Last note - everything works fine if I don't use Capistrano (eg just copy Rails app folder to server).. so I guess I am missing something on Capistrano side..

Answer 1

So it turns out that my redis folder had permissions problem and could not write to the database. So I checked where the database file is being saved (or rather not saved in my case) and did sudo chown redis:redis now all is good.