如何使用 nodejs 将消息发送到 RabbitMQ 队列

Question

I am currently building a load balancing tool to take different stress measurements of how servers handle http protocol requests. Currently, I can send just a ton, but I cannot process them all. That is why I am looking to put it all into a message queue like rabbit, using AMQP.

My current code https://github.com/yugely/Stork

I am currently using an event loop and a timeout to adequately do what I am intending to accomplish.

I want to use RabbitMQ by using one of my current loggers to "emit" a message into the message queue. I don't know how to modularize it so I don't have to constantly create channels as all the tutorials seem to just copy paste each other without going into how to use it in external files.

I'm hoping someone can either lead me to what I may be duping. I'm unsure of how to even ask this question. I have been looking at RabbitMQ and AMQP to handle a message queue for a project. The issue I am facing is that I don't know how to send a message to rabbit mq. Let me illustrate what I am understanding by doing a copypasta of the first tutorial on the rabbitmq site:

send.js

var amqp = require('amqplib/callback_api');

amqp.connect('amqp://localhost', function(error0, connection) {
    if (error0) {
        throw error0;
    }
    connection.createChannel(function(error1, channel) {
        if (error1) {
            throw error1;
        }

        var queue = 'hello';
        var msg = 'Hello World!';

        channel.assertQueue(queue, {
            durable: false
        });
        /*
         How do I do this outside the functions so receive gets it? How can I call a method/function to send a message to an existing queue through an existing channel?
        */
        channel.sendToQueue(queue, Buffer.from(msg));

        console.log(" [x] Sent %s", msg);
    });
    setTimeout(function() {
        connection.close();
        process.exit(0);
    }, 500);
});

receive.js

var amqp = require('amqplib/callback_api');

amqp.connect('amqp://localhost', function(error0, connection) {
    if (error0) {
        throw error0;
    }
    connection.createChannel(function(error1, channel) {
        if (error1) {
            throw error1;
        }

        var queue = 'hello';

        channel.assertQueue(queue, {
            durable: false
        });

        console.log(" [*] Waiting for messages in %s. To exit press CTRL+C", queue);
        /*
    How do I set up a consumer outside this function? 
         */
        channel.consume(queue, function(msg) {
            console.log(" [x] Received %s", msg.content.toString());
        }, {
            noAck: true
        });
    });
});

1. For the sender, do I just always create a new channel for every message?

2. All the tutorials accept arguments when you run the "node" terminal command. The only way I can currently see this working is to use the "child process" library, but that would be bad news bears right? As wouldn't that create another nodejs cluster?

3. If I have to use a client to send messages, am I able to use axios? (I've seen some people claiming they are able to but I don't know for sure). Since it will be using the amqp protocol, what is the amqp client?

4. Or are these queues like instantiated in the entry file? Like you set up the queue when you run your entry point command and then allow the different "events" to send messages to the queue?

How can I modularize this?

Just to illustrate, here is my current axios code

RadioFlyer.js

    await axios(flight.journal.actions[modkey]).then(function (response) {
        reaction.key.type = messageType.HTTPResponse.Okay
        reaction.message = response === undefined ?  response : "no response"
        let smaug = typeof reaction.message.status === "undefined" ?  reaction.message : reaction.message.status
        flight.journal.reactions.push(reaction)
        let pulse = {
            id: flight.id + "-" + index,
                timestamp: Date.now(),
            body: {
            payload: {
                protocol : reaction.protocol,
                    type: messageType.HTTPResponse.Okay,
                    url: flight.journal.actions[modkey].baseURL,
                    status: smaug
                }
            }
        }
       /*
        Can I emit a messaging event to my logger
        */
        //emit
        seidCar.HTTPLogger.emit("logHttp", reaction)
        //emit
        seidCar.HeartbeatLogger.emit("pulse", pulse)
    }).catch(function (error) {
      reaction.key.type = messageType.HTTPResponse.Error
      reaction.message = error.response === undefined ?  error.code : error.response

      let smaug = typeof reaction.message.status === "undefined" ?  reaction.message : reaction.message.status
      let pulse = {
            id: flight.id + "-" + index,
            timestamp: Date.now(),
            body: {
                payload: {
                    protocol : reaction.protocol,
                    type: messageType.HTTPResponse.Error,
                    url: flight.journal.actions[modkey].baseURL,
                    status: smaug
                }
            }
        }
      let err = {
          id: flight.id+"-"+index+"-ERROR",
          timestamp : Date.now(),
          fatal : false,
          potentialFix : "Examine Http Call with id: " + flight.id + "-" + index,
          body: {
              payload: {
                  protocol : reaction.protocol,
                  type: messageType.HTTPResponse.Error,
                  url: flight.journal.actions[modkey].baseURL,
                  status: smaug
              }
          }
      }
      flight.journal.reactions.push(reaction)
      //emit
      seidCar.HTTPLogger.emit("logHttp", reaction)
      //emit
      seidCar.ErrorLogger.emit("logError", err)
        //emit
        seidCar.HeartbeatLogger.emit("pulse", pulse)
    })

And have my logger handler the sending to the queue?

HTTPLogger.js

/*
Can I now send the message to the queue here, in this file?
*/
const HTTPEventLogger = require("events")
const emitter = new HTTPEventLogger()
const errorEmitter = require("./ErrorLogger").Emitter

class HTTPLogger extends HTTPEventLogger {
  logHttp(message) {
      switch (message.key.type) {
        case "ERROR":
          if (message !== undefined) {
            console.log(message)
          } else {
            errorEmitter.emit("tossIt", {
              error:"HTTP error Message is undefined in ~/Homestasis/Agent/HTTPLoggerjs.",
              poi:"check for recent additions to pilot agents in ~/Pilots/Agent",
              timestamp: Date.now(),
              potentialFix:"look to where you are emitting the message. Function Scope"
            })
          }
          break;
        case "OKAY":
          if (message !== undefined) {
            console.log(message)//bState.message.status)
          } else {
            errorEmitter.emit("tossIt", {
              error:"HTTP okay Message is undefined in ~/Homestasis/Agent/HTTPLoggerjs.",
              poi:"check for recent additions to pilot agents in ~/Pilots/Agent",
              timestamp: Date.now(),
              potentialFix:"look to where you are emitting the message. Function Scope"
            })
          }
          break;
        default:
          errorEmitter.emit("tossIt", "this is a tossIt error log. No http key type was caught bSate = " + bState)
      }
  }
}
var logger = new HTTPLogger()
emitter.on("logHttp",logger.logHttp)

exports.Emitter = emitter

Thats how I'd like to send the message.

I'd like to be able to receive it much the same way

I'm not sure how to implement it given how I perceive it currently working and I am missing a key part of the equation. Thank you for your time!

Answer 1

I decided to not use RabbitMQ.

For node, RSMQ is what I am going with. I realized I fundamentally needed to shift how I viewed message queues working practically to get them to work practically. I am using RSMQ, as I can understand how it fits into how I built my project out. I was going with ZeroMQ at first (I still may, the pliability of sockets on steroids). But, I do like the actual features and stuff when I actually think about using it like a "mainline nerve".

The way I want to build it out goes as such:

Axios Makes the call, fires off an event to the httplogger which fires off an event to the message queue. So:

RadioFlyer.js > HttpLogger.js > RSMQ-Messagequeue.js > next things > and next...

RSMQ-Messagequeue.js acts as a "middle man" or an intermediary to put the stops on the wheels when overloaded.

The pros of RSMQ is that I also intend to implement a Redis Cache (I could implement ZeroMQ and do much the same), but I like that RSMQ allows for more queue options, instead of 1 queue with many topics.

There is a more fleshed out answer.