Trace Error in Rack Middleware

Question

I'm not 100% sure at this point, but I believe I'm getting an exception that is causing the status of a request to respond with a 401.

I've eliminated all the code in the controller, and the issue still happens, and something seems to be catching the exception in order to return the 401 status, so there's no stack trace.

I've created a custom middleware to try and debug, but I still haven't been able to get to the source of the issue.

How can I isolate where the issue is happening?

We are using heroku, so pry doesn't seem to work, nor do I have command line access to the running server. I've not been able to replicate the issue on my local dev machine.

Answer 1

I never found out how to inject logging, but a bit of a hacky workaround is to create an additional middleware, which can perform logging where you want it.

class Debugger
  def initialize(app, *args)
    @app = app
    @args = args
  end

  def call(env)
    Rails.logger.debug "executing app.call Debugger #{@args.first}"
    status, headers, body = @app.call(env)

    Rails.logger.debug "Status: #{status} after app.call Debugger #{@args.first}"
    [status, headers, body]
  end
end

and then adding the logger where you want in the middleware stack, in Rails this would go in config/application.rb :

config.middleware.insert_before Rack::Sendfile, "Debugger", "1"

which would cause the Rails logger to add

Executing app.call Debugger 1

Then execute Rack:Sendfile, and the log:

Status: 400 after app.call Debugger 1

Assuming you are using rails, and the status code returned by Rack::Sendfile returned a 400 status code.

Note, you will need to modify the logger, and how you are configuring the middleware to run if you are not using Rails.

This is not an ideal solution, there's probably a better way to do it, but at the time of writing I don't have a better one.

Answer 2

An alternative is the following module which inserts a debug message at each middleware entry/return (with the status code printed after each return) (adapted from https://github.com/DataDog/dd-trace-rb/issues/368 )

# Trace each middleware module entry/exit. Freely adapted from
# https://github.com/DataDog/dd-trace-rb/issues/368 "Trace each middleware in the Rails stack"
module MiddlewareTracer
  def call(env)
    Rails.logger.debug { "MiddleWare: entry #{self.class.name}" }
    status, headers, response = super(env)
    Rails.logger.debug { "MiddleWare: #{self.class.name}, returning with status #{status}" }
    [status, headers, response]
  end
end

# Instrument the middleware stack after initialization so that we
# know the stack won't be changed afterwards.
Rails.configuration.after_initialize do
  Rails.application.middleware.each do |middleware|
    klass = middleware.klass

    # There are a few odd middlewares that aren't classes.
    klass.prepend(MiddlewareTracer) if klass.respond_to?(:prepend)
  end
end