Webpack Dev Server proxied GET returns empty JSON object when data is expected

Question

I'm running a local instance of FoundryVTT which, among a lot of other things, serve two resources:

1. http://localhost:30000/modules/_dev-mode/lang/en.json (5.6 kB JSON)
2. http://localhost:30000/lang/en.json (24.9 kB JSON)

I have setup webpack-dev-server to start a proxy with hot reload, which I want to use for development. I have enabled all logging I could find to troubleshoot the problem I'm experiencing:

devServer: {
  client: {
    logging: "verbose",
  },
  hot: true,
  devMiddleware: {
    writeToDisk: true,
  },
  static: false,
  proxy: {
    context: (pathname, req) => {
      // Don't proxy hot reload
      const proxy = !pathname.match("^/ws");
      console.log("[CONTEXT]", pathname, proxy);
      return proxy;
    },
    target: "http://localhost:30000",
    ws: true,
    logLevel: "debug",
    changeOrigin: true,
    //selfHandleResponse: true,
    onProxyReq: (proxyReq, req, res) => {
      try {
        const proxyHost = req?.headers?.host || "HOST?";
        const proxyUrl = `${req.protocol}://${proxyHost}${req.path}`;
        console.log(`[${req.method}] ${proxyUrl}`);
      } catch (error) {
        console.log("[ERROR]", "onProxyReq", error);
      }
    },
    onProxyRes: (proxyRes, req, res) => {
      try {
        const proxyHost = req?.headers?.host || "HOST?";
        const proxyUrl = `${req.protocol}://${proxyHost}${req.path}`;
        const targetHeader = proxyRes?.socket?._httpMessage?._header;
        const targetHost =
          /host:\s+(?<host>\S+)/.exec(targetHeader)?.groups?.host ||
          "HOST?";
        const url = `${proxyRes.req.protocol}//${targetHost}${proxyRes.req.path}`;
        const exchange = `[${req.method}] [${proxyRes.statusCode}] ${proxyUrl} -> ${url}`;
        console.log(exchange);
      } catch (error) {
        console.log("[ERROR]", "onProxyRes", error);
      }
    },
    onError: (err, req, res, target) => {
      console.log("[ERROR]", "onError", err, req, res, target);
    },
  },
},

I also run this with DEBUG = "express:*" to enable logging from Express.

Accessing the first resource (http://localhost:30000/modules/_dev-mode/lang/en.json) via the proxy (http://localhost:8080/modules/_dev-mode/lang/en.json) returns the JSON as expected.

The devServer configuration logs:

[CONTEXT] /modules/_dev-mode/lang/en.json true
[GET] http://localhost:8080/modules/_dev-mode/lang/en.json
[GET] [200] http://localhost:8080/modules/_dev-mode/lang/en.json -> http://localhost:30000/modules/_dev-mode/lang/en.json

The Express debugging logs:

express:router dispatching GET /modules/_dev-mode/lang/en.json +12m
express:router query  : /modules/_dev-mode/lang/en.json +1ms
express:router expressInit  : /modules/_dev-mode/lang/en.json +1ms
express:router compression  : /modules/_dev-mode/lang/en.json +1ms
express:router middleware  : /modules/_dev-mode/lang/en.json +0ms
express:router handler  : /modules/_dev-mode/lang/en.json +1ms

Accessing the second resource (http://localhost:30000/lang/en.json) via the proxy (http://localhost:8080/lang/en.json) does not work as expected. Instead of 24.9 kB it only returns {} .

The devServer configuration logs nothing in this case.

The Express debugging logs (notice the handler row from the other example is missing):

express:router dispatching GET /lang/en.json +4m
express:router query  : /lang/en.json +0ms      
express:router expressInit  : /lang/en.json +0ms
express:router compression  : /lang/en.json +1ms
express:router middleware  : /lang/en.json +0ms 

A guess is that the problem might be caused by the large JSON size and chunking. If I skip the proxy the entire JSON is returned as expected. This is how the requests headers look like in the no-proxy case:

GET /lang/en.json HTTP/1.1
Host: localhost:30000
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="99", "Google Chrome";v="99"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.82 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Sec-Fetch-Site: none
Sec-Fetch-Mode: navigate
Sec-Fetch-User: ?1
Sec-Fetch-Dest: document
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9,sv;q=0.8
Cookie: session=5ec3bb8586ae5062804c72ae

And the response headers:

HTTP/1.1 200 OK
X-Powered-By: Express
Cache-Control: no-cache
Accept-Ranges: bytes
Last-Modified: Sun, 06 Mar 2022 09:01:38 GMT
ETag: W/"15127-17f5e767fee"
Content-Type: application/json; charset=UTF-8
Vary: Accept-Encoding
Content-Encoding: gzip
Date: Mon, 21 Mar 2022 19:09:01 GMT
Connection: keep-alive
Keep-Alive: timeout=5
Transfer-Encoding: chunked

I feel like something is going wrong inside the proxy and I can't figure out what. Is there something obviously incorrect with my setup, or some way for me to get more logs or info about what's happening?

Answer 1

Debugged the proxy by enabling auto attach in Visual Studio Code.

The problem was that the proxy was serving the request to http://localhost:8080/lang/en.json using .../<project>/dist/lang/en.json instead of forwarding it to http://localhost:30000/lang/en.json .

In this case, only requests for resources under /modules/<project> should be served from the folder .../<project>/dist . Setting the publicPath solved this:

devServer: {
  devMiddleware: {
    publicPath: "/modules/<project>"
  }
},