nginx reverse proxy a REST service alternate 200 and 404 responses for same uri

Question

the nginx.conf:

   server {
    listen 8080;
   }
   server {
    listen       80;
    server_name  localhost;

    location / {
        root   /test/public;
        index  index.html index.htm;
    }

    location /api {
        proxy_pass http://localhost:8080;
    }

    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   html;
    }
}

the request and reponse headers are almost plain, no auth/session/cache parameters involved.

For same uri, first request will return successfully, while second will return 404, and so on. I've tried disabling proxy buffering, but has no effect.

Answer 1

I'm 99.99% sure you have IPv6 enabled. In that case localhost resolves into two IP addresses 127.0.0.1 and [::1] and nginx balancing requests between them.

http://nginx.org/r/proxy_pass :

If a domain name resolves to several addresses, all of them will be used in a round-robin fashion.

On the other hand, you have listen 8080; directive that tends to listens only to IPv4 addresses (depends on OS, nginx version and other environment).

You could solve you problem in several ways:

1. use explicit IPv4 address proxy_pass http://127.0.0.1:8080;
2. use explicit IPv4 and IPv6 listen listen [::]:8080 ipv6only=off;

Answer 2

I observed the same problem in a docker enviroment. But the reason was independed from nginx. I just made a stupid copy-paste-mistake.

The setting:

I deployed the docker container by several docker-compose files. So I have following structure:

• API-Gateway-Container based on nginx which references to
• Webserver 1 based on nginx and
• Webserver 2 based on nginx

Each of them has its own docker and docker-compose file. Because the structure of the compose-files for Webserver1 and Webserver2 is very similiar, I copied it and replaced the container name and some other stuff. So far so good. Starting and stopping the containers was no problem, watching them by docker container ls shows no abnormality. Accessing Webserver1 and Webserver2 by http://localhost:<Portnumber for server> was no problem, but accessing Webserver1 through the api gateway leads to alternating 200 and 404 responses, while Webserver2 works well.

After days of debugging I found the problem: As I mentioned I copied the docker-compose file from Webserver1 for Webserver2 and while I replaced the container name, I forgot to replace the service name. My docker-compose file starts like

version: '3'

services:  
  webserver1:
    image: 'nginx:latest'
   container_name: webserver2
...

This constellation also leads to the described behavior.

Hope, someone can save some days or hours by reading this post ;-)

André

Answer 3

Well. In my case, the problem was pretty straightforward. What was happening was, I had about 15 server blocks, and the port that I setup for my nodejs proxy_pass was already being used on some old server block hiding in my enabled servers directory. So nginx randomly was proxy passing to the old server which was not running and the one I just started.

So I just greped for the port number in the directory and found 2 instances. Changed my port number and the problem was fixed.