502 Bad Gateway HAproxy

Question

I have Ubuntu 12.04LTS running. My webserver is Tomcat 7.0.42 and I use HAProxy as proxy server. My application is a servlet application which uses websockets.

Sometime when I request my page I get "502 Bad Gateway" error on some resources not on all, but on some. I think that this has something to do with my HAProxy configuration, which is the following:

global
    maxconn     4096 # Total Max Connections. This is dependent on ulimit
    nbproc      1

defaults
    mode        http
    option  http-server-close
    option httpclose
#   option  redispatch
    no option checkcache  # test against 502 error

frontend all 0.0.0.0:80
    timeout client 86400000
    default_backend www_backend
    acl is_websocket hdr(Upgrade) -i WebSocket
    acl is_websocket hdr_beg(Host) -i ws

    use_backend socket_backend if is_websocket

    backend www_backend
        balance roundrobin
        option forwardfor # This sets X-Forwarded-For
        timeout server 30000
        timeout connect 4000
        server apiserver localhost:8080 weight 1 maxconn 1024 check

    backend socket_backend
        balance roundrobin
        option forwardfor # This sets X-Forwarded-For
        timeout queue 5000
        timeout server 86400000
        timeout connect 86400000
        server apiserver localhost:8080 weight 1 maxconn 1024 check

What do I have to change to prevent the 502 error?

Answer 1

First, enable haproxy logging. It will simply tell you why it is giving the 502's. My guess is that the backend "localhost:8080" is simply not able to keep up or is not able to get a connection within 4000ms "timeout connect 4000".

Answer 2

You may have exceeded some of the default limits in HAProxy. Try adding the following to global section:

tune.maxrewrite 4096
tune.http.maxhdr 202

Answer 3

Your should replace your defaults with these ones :

# Set balance mode
balance random
# Set http mode
mode http
# Set http keep alive mode (https://cbonte.github.io/haproxy-dconv/2.3/configuration.html#4)
option http-keep-alive
# Set http log format
option httplog
# Dont log empty line
option dontlognull
# Dissociate client from dead server
option redispatch
# Insert X-Forwarded-For header
option forwardfor

Don't use http-server-close, it is likely the cause of your problems.

Keep-alive will have a connection with client and server at both side. It is working fine with websockets as well.

And if you enable the check on the server you need to as well configure it with something like this :

# Enable http check
option httpchk
# Use server configuration
http-check connect default
# Use HEAD on / with HTTP/1.1 protocol for Host example.com
http-check send meth HEAD uri / ver HTTP/1.1 hdr Host example.com
# Expect status 200 to 399
http-check expect status 200-399