[英]Node.js Cluster architecture: how to scale master worker
I have build a Node.js built-in cluster architecture with master/worker configuration. 我已经构建了一个具有主/工配置的Node.js内置集群架构。 The application is using
express
to serve api and static files and it is deployed with Docker: 该应用程序使用
express
来提供api和静态文件,并使用Docker进行部署:
[D O C K E R: 8080] --- N ---> [W O R K E R: 3001 ] --- 1 ---> [M A S T E R: 3000]
I have N worker in Worker.js
and 1 master in master.js
. 我有在N-工人
Worker.js
和在1个主master.js
。 Master and worker share common modules, while the master has a core module that loads core services and exposes an api on PORT=3001
, a worker loads the other apis on PORT=3000
, where the Docker container has been bind. 主服务器和工作器共享通用模块,而主服务器有一个核心模块,用于加载核心服务并在
PORT=3001
上公开api,工作人员在PORT=3000
上加载另一个api,Docker容器已经绑定。 While a routing proxy on a Worker will forward requests to the Master in order to serve requests to core modules, the other requests are being server on 3000 directly. 虽然Worker上的路由代理会将请求转发给Master以便向核心模块提供请求,但其他请求直接是服务器3000。
The start script looks like 启动脚本看起来像
'use strict';
(function() {
/// node clustering
const cluster = require('cluster');
const numCPUs = require('os').cpus().length;
if (cluster.isMaster) { // master node
var masterConfig=require('./config/masterconfig.json');
// Fork workers.
var maxCPUs = process.env.WORKER_NUM || masterConfig.cluster.worker.num;
maxCPUs=(maxCPUs>numCPUs)?numCPUs:maxCPUs;
for (let i = 0; i < maxCPUs; i++) {
const worker=cluster.fork();
}
var MasterNode=require('./lib/master');
var master= new MasterNode(masterConfig);
master.start()
.then(done=> {
console.log(`Master ${process.pid} running on ${masterConfig.pubsub.node}`);
})
.catch(error=> { // cannot recover from master error
console.error(`Master ${process.pid} error`,error.stack);
process.exit(1);
});
}
else if (cluster.isWorker) { // worker node
var workerConfig=require('./config/workerconfig.json');
var WorkerNode=require('./lib/worker');
var worker= new WorkerNode(workerConfig);
worker.start()
.then(done=> {
console.log(`Worker ${process.pid} running on ${workerConfig.pubsub.node}`);
})
.catch(error=> { // worker error is recoverable
console.error(`Worker ${process.pid} error`,error.stack);
});
}
}).call(this);
I have the following question. 我有以下问题。
1) By defaults the cluster
module share the underlining HTTP connection uses a round-robin approach to serve requests - see here , where worker processes are spawned using the child_process.fork() . 1)默认情况下,
cluster
模块共享下划线HTTP连接使用循环方法来处理请求 - 请参阅此处 ,其中使用child_process.fork()生成工作进程。 I do not know if I can customize this method to distributing incoming connections. 我不知道我是否可以自定义此方法来分配传入连接。
2) So far, I serve static files, templates (like pig/swig) in a express web application on each Worker on PORT=3000
, thus meaning that I run static routes for the web app on each worker instance spawned. 2)到目前为止,我在
PORT=3000
上的每个Worker上的快速Web应用程序中提供静态文件,模板(如pig / swig),这意味着我在每个生成的工作器实例上运行Web应用程序的静态路由。 I'm not sure if this, in terms of memory occupation is the best approach. 我不确定这在内存占用方面是否是最佳方法。
3) Other clustering approach. 3)其他聚类方法。 I have asked about migrating this architecture to PM2, despite it seems to promising, I'm not sure it's the best option - see here for more details.
我已经问过将这个架构迁移到PM2,尽管看起来很有希望,我不确定它是最好的选择 - 请看这里了解更多细节。
The master should only care about starting the workers and shutting them down properly/watching out for signals from the host and responding accordingly. 主人应该只关心启动工人并正确关闭他们/注意来自主人的信号并做出相应的响应。 In my experience, I've had tricky bugs because I exposed an API on the master that should have been on a worker.
根据我的经验,我遇到了棘手的错误,因为我在主服务器上暴露了应该在工作者身上的API。
If you are planning to switch to PM2, PM2 will handle your master and you will need to move that code to the worker anyway (or at least that used to be the case) 如果您打算切换到PM2,PM2将处理您的主服务器,您无论如何都需要将该代码移动到工作者(或者至少是以前的情况)
Regarding your questions; 关于你的问题;
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.