Nodejs proxy request coalescing

Question

I'm running into an issue with my http-proxy-middleware stuff. I'm using it to proxy requests to another service which ie might resize images et al.

The problem is that multiple clients might call the method multiple times and thus create a stampede on the original service. I'm now looking into (what some services call request coalescing ie varnish) a solution that would call the service once, wait for the response and 'queue' the incoming requests with the same signature until the first is done, and return them all in a single go... This is different from 'caching' results due to the fact that I want to prevent calling the backend multiple times simultaneously and not necessarily cache the results.

I'm trying to find if something like that might be called differently or am i missing something that others have already solved someway... but i can't find anything...

As the use case seems pretty 'basic' for a reverse-proxy type setup, I would have expected alot of hits on my searches but since the problemspace is pretty generic i'm not getting anything...

Thanks!

Answer 1

A colleague of mine has helped my hack my own answer. It's currently used as a (express) middleware for specific GET -endpoints and basically hashes the request into a map, starts a new separate request. Concurrent incoming requests are hashed and checked and walked on the separate request callback and thus reused. This also means that if the first response is particularly slow, all coalesced requests are too

This seemed easier than to hack it into the http-proxy-middleware, but oh well, this got the job done :)

const axios = require('axios');
const responses = {};
module.exports = (req, res) => {
  const queryHash = `${req.path}/${JSON.stringify(req.query)}`;
  if (responses[queryHash]) {
    console.log('re-using request', queryHash);
    responses[queryHash].push(res);
    return;
  }

  console.log('new request', queryHash);
  const axiosConfig = {
    method: req.method,
    url: `[the original backend url]${req.path}`,
    params: req.query,
    headers: {}
  };
  if (req.headers.cookie) {
    axiosConfig.headers.Cookie = req.headers.cookie;
  }
  responses[queryHash] = [res];
  axios.request(axiosConfig).then((axiosRes) => {
    responses[queryHash].forEach((coalescingRequest) => {
      coalescingRequest.json(axiosRes.data);
    });
    responses[queryHash] = undefined;
  }).catch((err) => {
    responses[queryHash].forEach((coalescingRequest) => {
      coalescingRequest.status(500).json(false);
    });
    responses[queryHash] = undefined;
  });
};