[英]Node.js: work with slower third party
I'm working with Node.js since 5 years and from 2 years on big projects with this framework. 我从5年开始使用Node.js,从2年开始使用该框架从事大型项目。 For two years, I'm confronted to a problem: how to work asynchronously and faster with non-async third party applications who's stacks requests, like MySQL, MongoDB or Apache SolR ?
两年来,我遇到了一个问题:如何与处理堆栈请求的非异步第三方应用程序(如MySQL,MongoDB或Apache SolR)异步且更快地工作?
I'm used to work with promises and to prepared several promises requests, like this: 我习惯于处理诺言并准备一些诺言请求,例如:
const promises = []
for (let i = 0; i < 1000; i += 1) {
const promise = mySqlRequest()
promises.push(promise)
}
Promise.all(promises)
.then()
.catch()
This example will work but will send 1000 requests at the same time to MySQL server, who's will stacks these requests and become very slow, will consume very large quantity of RAM. 这个例子可以工作,但是会同时向MySQL服务器发送1000个请求,后者将这些请求堆叠起来并变得非常慢,将消耗大量RAM。
The best solution is to do only one big request, but in some case it's impossible and I'm forced to make recursive function, which comes down to be synchronous and slower. 最好的解决方案是只执行一个大请求,但是在某些情况下这是不可能的,因此我不得不执行递归函数,该函数归结为同步且速度较慢。
So, what the best way to work fast and asynchronous with Node.js and a stacking third party ? 那么,与Node.js和堆叠第三方快速异步工作的最佳方法是什么?
If sending all requests at once doesn't work and sending them one by one doesn't work either, you'd need something similar to a thread-pool where some arbitrary number of tasks execute simultaneously. 如果一次发送所有请求都行不通,一次发送一个请求也不行,那么您需要类似于线程池的线程池,在线程池中同时执行任意数量的任务。 This is easily implementable using promises, for example like this:
使用promise很容易实现,例如:
Promise.pooled = function(arr, num = 5) {
return new Promise(function(resolve, reject) {
var i = -1;
var error = false;
var end = function() {
num--;
if(num === 0) resolve();
}
var next = function() {
if(error) return;
i++;
if(i >= arr.length)
end();
else
arr[i]().then(next).catch(onerr);
}
var onerr = function() {
if(error) return
error = true
reject.call(arguments)
}
for(var j = 0; j < num; j++)
next()
});
}
What this allows you is pass an array of functions as first argument, those functions should take no parameter and return a promise. 这允许您将函数数组作为第一个参数传递,这些函数应不带参数并返回承诺。 It will then execute exactly
num
simultaneously. 然后它将同时精确执行
num
。 If one of the promises fail, it will fail its own promise aswell and stop executing (this is changeable easily). 如果其中一个承诺失败,它将也将失败其自己的承诺并停止执行(这很容易更改)。
Example: 例:
Promise.after = function(ms) { return new Promise(function(resolve) { setTimeout(resolve, ms) }); } Promise.pooled = function(arr, num = 5) { return new Promise(function(resolve, reject) { var i = -1; var error = false; var end = function() { num--; if(num === 0) resolve(); } var next = function() { if(error) return; i++; if(i >= arr.length) end(); else arr[i]().then(next).catch(onerr); } var onerr = function() { if(error) return error = true reject.call(arguments) } for(var j = 0; j < num; j++) next() }); } var test = [ afterH(1000), afterH(500), afterH(800), afterH(600), afterH(3000), afterH(300), afterH(900), afterH(2000), afterH(1500), afterH(900), afterH(700), afterH(600), afterH(700) ]; // helper function, returns a function which when invoked returns a promise function afterH(ms) { return function() { console.log("Starting one job") return Promise.after(ms); } } Promise.pooled(test, 3).then(function() {console.log("All jobs finished") }).catch(function() {console.log("Job failed")})
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.