Rails4 ActionController::InvalidAuthenticityToken error

Question

I have Rails4 application running in production, and my visitors run occasionally into ActionController::InvalidAuthenticityToken error, which I cant reproduce. I get 2-4 daily notifications from various forms, with no clear logic behind. The report I receive shows that authenticity_token submitted by the form is different from one kept in session. How is it possible? I managed to run into the problem myself few times, however it's impossible to reproduce, all of sudden authenticity_token for the form is different from the one stored in session and InvalidAuthenticityToken arises.
Any ideas where to start looking?

Example:

 Request:
-------------------------------

  * URL        : https://domain/signin
  * HTTP Method: POST
  * IP address : 113.96.xx.xx
  * Parameters : {"utf8"=>"✓", "authenticity_token"=>"MOh9JDE1AZ0CbIw/M33vfhjRShwzI6oqMhi8lk+n7OE=", "email"=>"xxxx@xxx", "password"=>"[FILTERED]", "commit"=>"Sign In", "controller"=>"clients", "action"=>"signin", "locale"=>"en"}

-------------------------------
Session:
-------------------------------

  * session id: [FILTERED]
  * data: {"_csrf_token"=>"QazCSVGeZlxEh83XTM+f5PkC/zopwCF96yV4duRats0="}

Update: Wanted to add that I'm serving the pages via two load balanced AWS EC2 instances, and store sessions in Redis ElastiCache instance

Answer 1

Any forms generated by Rails (ie with form_for and such, not with you putting <form> in the template) will have the anti-CSRF token added as a hidden field when it's necessary. If you wrote the form yourself and didn't include the CSRF hidden input, then Rails relies on the CSRF meta tag and JavaScript for things to work. So if you wrote your own form and you didn't include the hidden field and if the client's JavaScript doesn't work for whatever reason, you can get this error. Because the "client's JavaScript doesn't work for whatever reason" clause is a difficult one to detect and debug, I actually intentionally removed the CSRF meta tag on my site. That way if I forgot to include the hidden input, it'll break for everyone (fast fail), I'd find out about it immediately, and I could fix it. I would recommend you do this as well.

With that being said, I would recommend you look at the access log for these "visitors". Do you see anything odd?

• Do they access the page that has the form on it immediately before submitting it? If not, maybe they're bots or actual, real-deal CSRF attempts (that's what this is checking for, isn't it? :)).
• Did they load the form on one EC2 and end up submitting to another? If so, can you turn one EC2 off and see if the errors go away?
• Did they somehow lose their session? This could be your problem or theirs.

Answer 2

Since rails uses JavaScript to append the authenticity_token to rails forms, I would double-triple check that you don't have a runtime JS error based of of dynamic content that is causing this heisenbug. If a JS error we're to break the whole application.js file, your forms would be invalid. Is this possible?

Answer 3

To answer my own question, in case someone runs into the same issue, it seems like removing csrf_meta_tag from the header fixed the problem for us. I dont know why. Could be that rails javascript responsible for setting the auth_token was interfering in some way with our javascript and caused the problem, but my gut feeling is that it had to so something with cache, either on server or client side. Anyway, after removing csrf_meta_tag it seems like we got rid of the problem. Just make sure you're using the form_tag for all your forms.

Answer 4

That it happens on sign in makes me suspect something may be wrong on your sign out, triggering the issue for users that sign out and immediately back in again.

Signing out usually destroys the current user session and replaces it with a new one, invalidating the CSRF tag. Usually this isn't an issue, since the user gets redirected to another page containing the new CSRF tag.

I can see this being an issue if

• logout leads to a page that might be cached, either in the browser or a proxy
• logout is done through AJAX, which in rare instances fails to do the right thing on success, failing to update the CSRF tag

Answer 5

I had same trouble. Server: nginx + passenger

nginx.conf:

http {
    ...
    expires    90d;
    ...
    server {
        server_name domain1.com
        ...
    }
    server {
        server_name domain2.com
        ...
    }
    server {
        server_name domain_3_with_rails.com
        ...
    }
}

Problems in instruction "expires 90d;" Browser locally cached page with form and with authenticity_token.

Solution: add "expires 0d;" for rails domains:

nginx.conf:

http {
    ...
    expires    90d;
    ...
    server {
        server_name domain1.com
        ...
    }
    server {
        server_name domain2.com
        ...
    }
    server {
        server_name domain_3_with_rails.com
        expires 0d;
        ...
    }
}

After that, be sure to restart Nginx.

For those who have apache: apache it certainly has a similar statement "expires" for nginx

Answer 6

I have the same issue as of now. I have been poking around and I noticed that if I turn cookies off (block the domain to be able to use cookies) I will run into a ActionController::InvalidAuthenticityToken everytime I do a POST .

So the user has JS enabled but does not allow cookies.

AFAIK the anti-CSRF token in Rails is sent as a session cookie server side, and then fails since the cookie could not be set.