Rails 3 query string

Question

Very weird error here. We correctly get a request for something like "/users/8788234"

In Rails we call:

redirect_to(:controller => 'login', :from_external_page => true, :on_login => request.env['REQUEST_URI']) and return

We see (as we'd expect) in the Rails log: Redirected to

https://sampleapp.com/login?from_external_page=true&on_login=%2Fusers%2F8788234

But then the next request we see from the IP has the values of the query string scrambled:

Started GET "/login?from_external_page=gehr&on_login=%2Shfref%2S8788234" for xx.xxx.xxx.xxx at yyyy-mm-dd

This both makes the query string values meaningless and causes the following error:

ArgumentError: invalid %-encoding

(The %2F was changed to %2S which is invalid). Every single value of each key-value pair within the query string is getting shifted by 13 characters. Every time we've seen this, the user agent reads: "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)", but we also see that user agent successfully navigate the application. Has anyone ever seen anything like this? http://www.whatismybrowser.com/ tells me that this user agent is IE9 running on Windows 7, but we haven't been able to reproduce the bug.

Answer 1

This certainly is encoding issue. I am using Rails 4 now and following is sample query string of my current project. Note the very first parameter in querystring is "utf8=✓" which is missing in your querystring.

profiles?utf8=✓&min_age=1&max_age=99&min_height=1&max_height=6&min_weight=1&max_weight=400

Try adding "# encoding: UTF-8 at the beginning of the file"

Answer 2

If anyone is curious, I ended up just writing a middleware parser for the nested query string that would rotate it back 13 characters in the event that a 13 character shift would make it valid. I'm not going to accept it as an answer, in the hopes that someone might actually be able to answer it. Anyway, here's my approach:

# used to parse out invalid query strings, attempt to fix them, and
# handle the resulting query appropriately
class InvalidEncodingParser

  # creates the middleware instance with a reference to the application
  def initialize(app)
    @app = app
  end

  # parse out bad queries, attempt to fix
  def call(env)
    begin
      # no need to scrub_nested_query if QUERY_STRING is blank or unescapable
      env['QUERY_STRING'].blank? or Rack::Utils.unescape(env['QUERY_STRING'])
    rescue ArgumentError
      env['QUERY_STRING'] = scrub_nested_query(env['QUERY_STRING'])
    end
    @app.call(env)
  end

  private

  # attempts to unescape both the query params and rot13 of the query params
  def scrub_nested_query(query_string = '')
    params = []
    (query_string || '').split(/[&;] */n).each do |param|
      if valid_query_param?(param)
        params << param
      elsif valid_query_param?(rotate_13_characters(param))
        params << rotate_13_characters(param)
      else
        raise ArgumentError
      end
    end
    params.join('&')
  end

  # applies a caesar cipher with a shift of 13 characters to the value of the
  # query key, value pair if the given param contains an equal sign
  def rotate_13_characters(param)
    key, value = param.split('=', 2)
    value.nil? ? param : (key + '=' + value.tr('A-Za-z', 'N-ZA-Mn-za-m'))
  end

  # attempts to unescape the param, returns false if it fails
  def valid_query_param?(param)
    param.blank? or Rack::Utils.unescape(param).present?
  rescue ArgumentError
    false
  end

end

... and then put the following at the bottom of my application.rb file

# Use invalid encoding middleware to parse out invalid encodings in the query string 
# of the url and handle them appropriately 
config.middleware.insert_before(ActionDispatch::ParamsParser, 'InvalidEncodingParser')