How can I make a custom reverse-proxy (preferrably a Node.js one) with good performance?

Question

I am making a chat-bot that is based on a HTTP Webhook API that sends requests via POST with JSON message in it's body. It runs in multithread across multiple servers. So here's the problem - when users send multiple commands at the same time, commands are being handled asynchronously, resulting in a mess - for example, if you send a command that increments some counter in database (I'm using MongoDB) twice at the same moment, the bot will answer twice with the same counter value, incrementing it only once in database.

So I've came up with an idea of some custom reverse-proxy with a queueing logic in it. This proxy would accept an HTTP request, transfer it to a chat-bot thread and remember the chat where the message came from. If another request will come from the same chat, it will put the request in some sort of queue and transfer it to the bot when the first one will complete.

I made this proxy on Node.js using uWebSockets.js as a HTTP server and native http.request as HTTP client to transfer requests, but it does not perform very well. When it's being used at the "real" load (2-3 requests/sec) it starts to spit out errors and the whole chatbot becomes unresponsive.

There's nothing special about request sending code - it just makes a request and then responses with status, headers and body that it got from backend.
In general these errors do occur:

Error: connect ECONNREFUSED x.x.x.x:3xxx
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1056:14) {
  errno: 'ECONNREFUSED',
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: 'x.x.x.x',
  port: 3xxx
}

Error: socket hang up
    at connResetException (internal/errors.js:559:14)
    at Socket.socketCloseListener (_http_client.js:376:25)
    at Socket.emit (events.js:208:15)
    at TCP.<anonymous> (net.js:588:12) {
  code: 'ECONNRESET'
}

I would like to hear any ideas about how can I fix my reverse-proxy or how can I use a ready one. Maybe there is a way to make a queueing mechanism without a custom reverse-proxy at all. I wanted to try bouncy module, but it seems to be deprecated with the last commit on GitHub made at 2014.

Answer 1

try this

// include dependencies
var express = require('express')
var proxy = require('http-proxy-middleware')

// proxy middleware options
var options = {
  target: 'http://www.example.org', // target host
  changeOrigin: true, // needed for virtual hosted sites
  ws: true, // proxy websockets
  pathRewrite: {
    '^/api/old-path': '/api/new-path', // rewrite path
    '^/api/remove/path': '/path' // remove base path
  },
  router: {
    // when request.headers.host == 'dev.localhost:3000',
    // override target 'http://www.example.org' to 'http://localhost:8000'
    'dev.localhost:3000': 'http://localhost:8000'
  }
}

// create the proxy (without context)
var exampleProxy = proxy(options)

// mount `exampleProxy` in web server
var app = express()
app.use('/api', exampleProxy)
app.listen(3000)

Answer 2

Use the Caching technique. This is likely because your server processes too much code and leads the socket server to fail.