[英]How to make a Redis (Bull) queue in a Node.js environment that's deployed to Heroku?
I've looked through all of the documentation, and I'm having some trouble finding examples of the correct implementation.我浏览了所有文档,但在找到正确实现的示例时遇到了一些麻烦。 First of all, I have the Heroku Redis add-on as you can see below:
首先,我有 Heroku Redis 附加组件,如下所示:
https://i.stack.imgur.com/9kcXW.png https://i.stack.imgur.com/9kcXW.png
I installed Redis and Bull to my Node.js environment and set up Redis as follows:我将 Redis 和 Bull 安装到我的 Node.js 环境并设置 Redis 如下:
const redis = require('redis');
const Queue = require('bull');
const redisClient = redis.createClient(process.env.REDIS_URL, {
tls: {
rejectUnauthorized: false
}
});
const bullQueue = new Queue('queue', process.env.REDIS_URL)
I'm trying to run the below function as a background task (this is what Heroku recommends for functions that take longer than 0.5 seconds to complete):我正在尝试将以下 function 作为后台任务运行(这是 Heroku 对于需要超过 0.5 秒才能完成的功能的建议):
app.post('/', async function(request, response) {
const client = await pool.connect() //pool is just a node-postgres pool
try {
await client.query('BEGIN')
let data = await function1(argumentList);
await function2(data);
await client.query('COMMIT')
} catch(err) {
await client.query('ROLLBACK')
console.log(err)
}
try {
await client.query('BEGIN')
const setOfItems = await function3()
var array = []
for (item of setOfItems) {
if (conditional) {
array.push(item)
}
}
await client.query('COMMIT')
let job = await bullQueue.add()
response.send(JSON.stringify(array))
} catch(err) {
await client.query('ROLLBACK')
console.log(err)
}
});
This function does some web scraping, database calls, and other things, so like I said it takes a few seconds to complete.这个 function 做了一些 web 抓取、数据库调用和其他事情,所以就像我说的需要几秒钟才能完成。 How should I change my function to add the request to the background queue and then return the JSON.stringified 'array' back to the user.
我应该如何更改我的 function 以将请求添加到后台队列,然后将 JSON.stringified 'array' 返回给用户。 Sorry if this is a noob Redis question, but I've looked at all the docs and I really am not sure how to proceed.
抱歉,如果这是一个菜鸟 Redis 问题,但我查看了所有文档,我真的不知道如何继续。
For starters, const redis = require('redis');
对于初学者,
const redis = require('redis');
isn't necessary.没有必要。 Bull connects to Redis under the hood, so you need only provide
process.env.REDIS_URL
for Heroku as you're doing. Bull 在引擎盖下连接到 Redis,因此您只需为 Heroku 提供
process.env.REDIS_URL
即可。 ( heroku config | grep REDIS
returns the endpoint URLs if needed) (
heroku config | grep REDIS
如果需要,将返回端点 URL)
Currently, all of your database code isn't delegated to the task queue at all.目前,您的所有数据库代码都没有委托给任务队列。 Try registering a processing function associated with the queue using
尝试注册与队列关联的处理 function 使用
bullQueue.process(job => {/* your database work goes here */})
The parameter job
needs to be populated with whatever serializable data your worker needs to do its job.参数
job
需要填充您的工作人员完成其工作所需的任何可序列化数据。 You populate this parameter using您使用填充此参数
bullQueue.add(/* job data goes here */)
A high-level approach might be something like:高级方法可能类似于:
const Queue = require('bull');
const bullQueue = new Queue('queue', process.env.REDIS_URL)
bullQueue.process(async job => {
console.log(job.data.argumentList);
/* do SQL stuff */
return [1, 2, 3, 4];
});
app.post('/', async (request, response) => {
const job = await bullQueue.add({argumentList: ['some', 'data']});
const array = await job.finished();
response.send(JSON.stringify(array))
});
Many other strategies exist for getting the results.存在许多其他策略来获得结果。 See returning job completions for some ideas.
有关一些想法,请参阅返回工作完成。
See also:也可以看看:
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.