Rails API Worker on Heroku with Sidekiq

Question

I'm working on setting up a Worker in a Rails API app to send emails through the SendGrid API after form submission. I've been using the following tutorial as a guide.

https://itnext.io/sidekiq-overview-and-how-to-deploy-it-to-heroku-b8811fea9347

When I try to submit the form, I get a response immediately - but no emails are sent. Any help would be greatly appreciated! Thanks.

app/controllers/leads_controller.rb

...

def create
    @lead = Lead.new(lead_params)
    if @lead.save
        @venues = Venue.all
        HardWorker.perform_async(@venues, @lead)
        render json: @lead, status: :created, location: @lead
    else
        render json: @lead.errors, status: :unprocessable_entity
    end
end

...

app/workers/hard_worker.rb

class HardWorker
    include Sidekiq::Worker
    sidekiq_options retry: false

    def perform(venues, lead)
        @venues = venues
        @lead = lead

        @venues.each do |venue|
            if venue.email
                UserNotifierMailer.send_lead_email(venue, @lead).deliver
            end
        end

        UserNotifierMailer.send_signup_email(@lead).deliver
    end
end

Procfile

web: bundle exec rails s
release: bin/rake db:migrate
hardworker: bundle exec sidekiq -c 2

Update with heroku logs

2019-05-14T16:07:19.832606+00:00 app[hardworker.1]: 4 TID-gt4ojjes8 HardWorker JID-69a69142b893a183306ddb69 INFO: start
2019-05-14T16:07:19.833556+00:00 app[hardworker.1]: 4 TID-gt4ojjes8 HardWorker JID-69a69142b893a183306ddb69 INFO: fail: 0.001 sec
2019-05-14T16:07:19.843658+00:00 app[hardworker.1]: 4 TID-gt4ojjes8 WARN: {"context":"Job raised exception","job":{"class":"HardWorker","args":["#<Venue::ActiveRecord_Relation:0x00005583e0e39680>","#<Lead:0x00007fbfa0114880>"],"retry":false,"queue":"default","jid":"69a69142b893a183306ddb69","created_at":1557850039.8114436,"enqueued_at":1557850039.8117154},"jobstr":"{\"class\":\"HardWorker\",\"args\":[\"#<Venue::ActiveRecord_Relation:0x00005583e0e39680>\",\"#<Lead:0x00007fbfa0114880>\"],\"retry\":false,\"queue\":\"default\",\"jid\":\"69a69142b893a183306ddb69\",\"created_at\":1557850039.8114436,\"enqueued_at\":1557850039.8117154}"}
2019-05-14T16:07:19.864496+00:00 app[hardworker.1]: 4 TID-gt4ojjes8 WARN: NoMethodError: undefined method `each' for "#<Venue::ActiveRecord_Relation:0x00005583e0e39680>":String
2019-05-14T16:07:19.882424+00:00 app[hardworker.1]: 4 TID-gt4ojjes8 WARN: /app/app/workers/hard_worker.rb:9:in `perform'
2019-05-14T16:07:19.882428+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:192:in `execute_job'
2019-05-14T16:07:19.882441+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:165:in `block (2 levels) in process'
2019-05-14T16:07:19.882444+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
2019-05-14T16:07:19.882445+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:133:in `invoke'
2019-05-14T16:07:19.882447+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:164:in `block in process'
2019-05-14T16:07:19.882448+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:137:in `block (6 levels) in dispatch'
2019-05-14T16:07:19.882450+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:109:in `local'
2019-05-14T16:07:19.882452+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:136:in `block (5 levels) in dispatch'
2019-05-14T16:07:19.882453+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:43:in `block in call'
2019-05-14T16:07:19.882455+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.1/lib/active_support/execution_wrapper.rb:87:in `wrap'
2019-05-14T16:07:19.882457+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.1/lib/active_support/reloader.rb:73:in `block in wrap'
2019-05-14T16:07:19.882458+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.1/lib/active_support/execution_wrapper.rb:87:in `wrap'
2019-05-14T16:07:19.882459+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.1/lib/active_support/reloader.rb:72:in `wrap'
2019-05-14T16:07:19.882461+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:42:in `call'
2019-05-14T16:07:19.882463+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:132:in `block (4 levels) in dispatch'
2019-05-14T16:07:19.882464+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:250:in `stats'
2019-05-14T16:07:19.882465+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:127:in `block (3 levels) in dispatch'
2019-05-14T16:07:19.882467+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/job_logger.rb:8:in `call'
2019-05-14T16:07:19.882469+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:126:in `block (2 levels) in dispatch'
2019-05-14T16:07:19.882470+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:74:in `global'
2019-05-14T16:07:19.882472+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:125:in `block in dispatch'
2019-05-14T16:07:19.882473+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:48:in `with_context'
2019-05-14T16:07:19.882475+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:42:in `with_job_hash_context'
2019-05-14T16:07:19.882476+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:124:in `dispatch'
2019-05-14T16:07:19.882478+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:163:in `process'
2019-05-14T16:07:19.882480+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:83:in `process_one'
2019-05-14T16:07:19.882481+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:71:in `run'
2019-05-14T16:07:19.882483+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:16:in `watchdog'
2019-05-14T16:07:19.882484+00:00 app[hardworker.1]: /app/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:25:in `block in safe_thread'
2019-05-14T16:10:24.013315+00:00 heroku[hardworker.1]: Restarting
2019-05-14T16:10:24.025034+00:00 heroku[hardworker.1]: State changed from up to starting
2019-05-14T16:10:24.930345+00:00 heroku[hardworker.1]: Stopping all processes with SIGTERM
2019-05-14T16:10:24.942407+00:00 app[hardworker.1]: 4 TID-gt4nwe3lw INFO: Shutting down
2019-05-14T16:10:24.942431+00:00 app[hardworker.1]: 4 TID-gt4nwe3lw INFO: Terminating quiet workers
2019-05-14T16:10:24.942534+00:00 app[hardworker.1]: 4 TID-gt4ojji50 INFO: Scheduler exiting...
2019-05-14T16:10:25.442840+00:00 app[hardworker.1]: 4 TID-gt4nwe3lw INFO: Pausing to allow workers to finish...
2019-05-14T16:10:26.444798+00:00 app[hardworker.1]: 4 TID-gt4nwe3lw INFO: Bye!
2019-05-14T16:10:26.590745+00:00 heroku[hardworker.1]: Process exited with status 0
2019-05-14T16:10:40.274601+00:00 heroku[hardworker.1]: Starting process with command `bundle exec sidekiq -c 2`
2019-05-14T16:10:40.864028+00:00 heroku[hardworker.1]: State changed from starting to up
2019-05-14T16:10:52.096992+00:00 app[hardworker.1]: 4 TID-gs793pw3g INFO: Running in ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
2019-05-14T16:10:52.097024+00:00 app[hardworker.1]: 4 TID-gs793pw3g INFO: See LICENSE and the LGPL-3.0 for licensing details.
2019-05-14T16:10:52.097026+00:00 app[hardworker.1]: 4 TID-gs793pw3g INFO: Upgrade to Sidekiq Pro for more features and support: http://sidekiq.org
2019-05-14T16:10:52.097208+00:00 app[hardworker.1]: 4 TID-gs793pw3g INFO: Booting Sidekiq 5.2.7 with redis options {:id=>"Sidekiq-server-PID-4", :url=>"redis://redistogo:REDACTED@pearlfish.redistogo.com:9778/"}
2019-05-14T16:10:52.115881+00:00 app[hardworker.1]: 4 TID-gs793pw3g WARN: Sidekiq 6.0 will require Redis 4.0+, you are using Redis v3.2.12
2019-05-14T16:10:52.116074+00:00 app[hardworker.1]: 4 TID-gs793pw3g INFO: Starting processing, hit Ctrl-C to stop

Answer 1

Ah, you're enqueueing HardWorker with @venues , which is an ActiveRecord::Relation object that gets serialized to a string by Sidekiq.

When enqueueing Sidekiq jobs, you want to pass the primary keys of records instead of the records themselves. ActiveJob abstracts this away, but when using Sidekiq directly, never pass ActiveRecord objects. Use primitives instead.

In your case, I think you want something like this:

# controller
HardWorker.perform_async(@lead.id)

# worker
def perform(lead_id)
  @venues = Venue.all
  @lead = Lead.find(lead_id)
  # ...
end

Also, I strongly recommend setting up error tracking with Sentry or Honeybadger so you're notified when exceptions like these are encountered.

Hope this helps!