[英]polling with delayed_job
I have a process which takes generally a few seconds to complete so I'm trying to use delayed_job to handle it asynchronously. 我有一个过程,通常需要几秒钟才能完成,因此我尝试使用delayed_job来异步处理它。 The job itself works fine, my question is how to go about polling the job to find out if it's done.
工作本身运作正常,我的问题是如何轮询工作以确定是否已完成。
I can get an id from delayed_job by simply assigning it to a variable: 我可以通过简单地将它分配给变量来获取delayed_job的id:
job = Available.delay.dosomething(:var => 1234) job = Available.delay.dosomething(:var => 1234)
+------+----------+----------+------------+------------+-------------+-----------+-----------+-----------+------------+-------------+
| id | priority | attempts | handler | last_error | run_at | locked_at | failed_at | locked_by | created_at | updated_at |
+------+----------+----------+------------+------------+-------------+-----------+-----------+-----------+------------+-------------+
| 4037 | 0 | 0 | --- !ru... | | 2011-04-... | | | | 2011-04... | 2011-04-... |
+------+----------+----------+------------+------------+-------------+-----------+-----------+-----------+------------+-------------+
But as soon as it completes the job it deletes it and searching for the completed record returns an error: 但是一旦完成作业,它就会删除它,并且搜索完成的记录会返回错误:
@job=Delayed::Job.find(4037)
ActiveRecord::RecordNotFound: Couldn't find Delayed::Backend::ActiveRecord::Job with ID=4037
@job= Delayed::Job.exists?(params[:id])
Should I bother to change this, and maybe postpone the deletion of complete records? 我是否需要改变这一点,并推迟删除完整的记录? I'm not sure how else I can get a notification of it's status.
我不知道我怎么能得到它的状态通知。 Or is polling a dead record as proof of completion ok?
或者正在查看死记录作为完成证明吗? Anyone else face something similar?
其他人面对类似的事情?
Let's start with the API. 让我们从API开始吧。 I'd like to have something like the following.
我想要像下面这样的东西。
@available.working? # => true or false, so we know it's running
@available.finished? # => true or false, so we know it's finished (already ran)
Now let's write the job. 现在让我们写下这份工作。
class AwesomeJob < Struct.new(:options)
def perform
do_something_with(options[:var])
end
end
So far so good. 到现在为止还挺好。 We have a job.
我们有一份工作。 Now let's write logic that enqueues it.
现在让我们编写将其排列的逻辑。 Since Available is the model responsible for this job, let's teach it how to start this job.
由于Available是负责这项工作的模型,让我们教它如何开始这项工作。
class Available < ActiveRecord::Base
def start_working!
Delayed::Job.enqueue(AwesomeJob.new(options))
end
def working?
# not sure what to put here yet
end
def finished?
# not sure what to put here yet
end
end
So how do we know if the job is working or not? 那么我们如何知道这项工作是否有效? There are a few ways, but in rails it just feels right that when my model creates something, it's usually associated with that something.
有几种方法,但在rails中,我觉得正确的是,当我的模型创建某些东西时,它通常与那些东西相关联。 How do we associate?
我们如何联想? Using ids in database.
在数据库中使用id。 Let's add a
job_id
on Available model. 让我们在Available模型上添加一个
job_id
。
While we're at it, how do we know that the job is not working because it already finished, or because it didn't start yet? 虽然我们正在努力,但我们怎么知道这项工作因为已经完成而无法工作,或者因为它还没有开始? One way is to actually check for what the job actually did.
一种方法是实际检查作业实际上做了什么。 If it created a file, check if file exists.
如果它创建了一个文件,请检查文件是否存在。 If it computed a value, check that result is written.
如果计算了一个值,请检查结果是否已写入。 Some jobs are not as easy to check though, since there may be no clear verifiable result of their work.
有些工作并不容易检查,因为他们的工作可能没有明确的可验证结果。 For such case, you can use a flag or a timestamp in your model.
对于这种情况,您可以在模型中使用标志或时间戳。 Assuming this is our case, let's add a
job_finished_at
timestamp to distinguish a not yet ran job from an already finished one. 假设这是我们的情况,让我们添加一个
job_finished_at
时间戳来区分尚未运行的作业和已经完成的作业。
class AddJobIdToAvailable < ActiveRecord::Migration
def self.up
add_column :available, :job_id, :integer
add_column :available, :job_finished_at, :datetime
end
def self.down
remove_column :available, :job_id
remove_column :available, :job_finished_at
end
end
Alright. 好的。 So now let's actually associate
Available
with its job as soon as we enqueue the job, by modifying the start_working!
现在让我们联想实际
Available
,通过修改只要我们排队的工作,其工作start_working!
method. 方法。
def start_working!
job = Delayed::Job.enqueue(AwesomeJob.new(options))
update_attribute(:job_id, job.id)
end
Great. 大。 At this point I could've written
belongs_to :job
, but we don't really need that. 在这一点上,我可以编写
belongs_to :job
,但我们并不真的需要它。
So now we know how to write the working?
那么现在我们知道如何编写
working?
method, so easy. 方法,这么容易。
def working?
job_id.present?
end
But how do we mark the job finished? 但是我们如何标记完成的工作呢? Nobody knows a job has finished better than the job itself.
没有人知道工作比工作本身更好。 So let's pass
available_id
into the job (as one of the options) and use it in the job. 因此,让我们将
available_id
传递给作业(作为其中一个选项)并在作业中使用它。 For that we need to modify the start_working!
为此,我们需要修改
start_working!
method to pass the id. 传递id的方法。
def start_working!
job = Delayed::Job.enqueue(AwesomeJob.new(options.merge(:available_id => id))
update_attribute(:job_id, job.id)
end
And we should add the logic into the job to update our job_finished_at
timestamp when it's done. 我们应该将逻辑添加到作业中,以便在完成后更新
job_finished_at
时间戳。
class AwesomeJob < Struct.new(:options)
def perform
available = Available.find(options[:available_id])
do_something_with(options[:var])
# Depending on whether you consider an error'ed job to be finished
# you may want to put this under an ensure. This way the job
# will be deemed finished even if it error'ed out.
available.update_attribute(:job_finished_at, Time.current)
end
end
With this code in place we know how to write our finished?
有了这个代码,我们知道如何编写
finished?
method. 方法。
def finished?
job_finished_at.present?
end
And we're done. 我们已经完成了。 Now we can simply poll against
@available.working?
现在我们可以简单地对
@available.working?
轮询@available.working?
and @available.finished?
和
@available.finished?
Also, you gain the convenience of knowing which exact job was created for your Available by checking @available.job_id
. 此外,通过检查
@available.job_id
,您可以方便地了解为您的可用作业创建了哪个确切的作业。 You can easily turn it into a real association by saying belongs_to :job
. 您可以通过说
belongs_to :job
轻松将其转换为真正的关联。
I ended up using a combination of Delayed_Job with an after(job) callback which populates a memcached object with the same ID as the job created. 我最终使用了Delayed_Job和after(job)回调的组合,它使用与创建的作业相同的ID填充memcached对象。 This way I minimize the number of times I hit the database asking for the status of the job, instead polling the memcached object.
这样,我最小化了数据库询问作业状态的次数,而不是轮询memcached对象。 And it contains the entire object I need from the completed job, so I don't even have a roundtrip request.
它包含我完成的作业所需的整个对象,所以我甚至没有往返请求。 I got the idea from an article by the github guys who did pretty much the same thing.
我从github的一篇文章中得到了这个想法,他们做了几乎相同的事情。
https://github.com/blog/467-smart-js-polling https://github.com/blog/467-smart-js-polling
and used a jquery plugin for the polling, which polls less frequently, and gives up after a certain number of retries 并使用jquery插件进行轮询,轮询次数较少,并在经过一定次数的重试后放弃
https://github.com/jeremyw/jquery-smart-poll https://github.com/jeremyw/jquery-smart-poll
Seems to work great. 似乎工作得很好。
def after(job)
prices = Room.prices.where("space_id = ? AND bookdate BETWEEN ? AND ?", space_id.to_i, date_from, date_to).to_a
Rails.cache.fetch(job.id) do
bed = Bed.new(:space_id => space_id, :date_from => date_from, :date_to => date_to, :prices => prices)
end
end
I think that the best way would be to use the callbacks available in the delayed_job. 我认为最好的方法是使用delayed_job中提供的回调。 These are: :success, :error and :after.
这些是:成功,:错误和:之后。 so you can put some code in your model with the after:
所以你可以在你的模型中添加一些代码:
class ToBeDelayed
def perform
# do something
end
def after(job)
# do something
end
end
Because if you insist of using the obj.delayed.method, then you'll have to monkey patch Delayed::PerformableMethod and add the after
method there. 因为如果你坚持使用obj.delayed.method,那么你将不得不修补Delayed :: PerformableMethod并
after
那里添加after
方法。 IMHO it's far better than polling for some value which might be even backend specific (ActiveRecord vs. Mongoid, for instance). 恕我直言,它远比轮询某些可能甚至特定于后端的值更好(例如,ActiveRecord与Mongoid)。
The simplest method of accomplishing this is to change your polling action to be something similar to the following: 实现此目的的最简单方法是将您的轮询操作更改为类似于以下内容:
def poll
@job = Delayed::Job.find_by_id(params[:job_id])
if @job.nil?
# The job has completed and is no longer in the database.
else
if @job.last_error.nil?
# The job is still in the queue and has not been run.
else
# The job has encountered an error.
end
end
end
Why does this work? 为什么这样做? When
Delayed::Job
runs a job from the queue, it deletes it from the database if successful . 当
Delayed::Job
从队列中运行作业时, 如果成功 ,它将从数据库中删除它。 If the job fails, the record stays in the queue to be ran again later, and the last_error
attribute is set to the encountered error. 如果作业失败,则记录将保留在队列中以便稍后再次运行,并且
last_error
属性将设置为遇到的错误。 Using the two pieces of functionality above, you can check for deleted records to see if they were successful. 使用上面的两个功能,您可以检查已删除的记录以查看它们是否成功。
The benefits to the method above are: 上述方法的好处是:
You can encapsulate this functionality in a model method by doing something like the following: 您可以通过执行以下操作将此功能封装在模型方法中:
# Include this in your initializers somewhere
class Queue < Delayed::Job
def self.status(id)
self.find_by_id(id).nil? ? "success" : (job.last_error.nil? ? "queued" : "failure")
end
end
# Use this method in your poll method like so:
def poll
status = Queue.status(params[:id])
if status == "success"
# Success, notify the user!
elsif status == "failure"
# Failure, notify the user!
end
end
I'd suggest that if it's important to get notification that the job has completed, then write a custom job object and queue that rather than relying upon the default job that gets queued when you call Available.delay.dosomething
. 我建议如果获得作业已完成的通知很重要,那么编写一个自定义作业对象并排队,而不是依赖于在调用
Available.delay.dosomething
时排队的默认作业。 Create an object something like: 创建一个像这样的对象:
class DoSomethingAvailableJob
attr_accessor options
def initialize(options = {})
@options = options
end
def perform
Available.dosomething(@options)
# Do some sort of notification here
# ...
end
end
and enqueue it with: 并将其排列:
Delayed::Job.enqueue DoSomethingAvailableJob.new(:var => 1234)
The delayed_jobs table in your application is intended to provide the status of running and queued jobs only. 应用程序中的delayed_jobs表旨在仅提供运行和排队作业的状态。 It isn't a persistent table, and really should be as small as possible for performance reasons.
它不是一个持久表,并且出于性能原因,它应该尽可能小。 Thats why the jobs are deleted immediately after completion.
这就是为什么工作在完成后立即被删除。
Instead you should add field to your Available
model that signifies that the job is done. 相反,您应该在
Available
模型中添加字段,表示作业已完成。 Since I'm usually interested in how long the job takes to process, I add start_time and end_time fields. 由于我通常对作业处理所需的时间感兴趣,因此我添加了start_time和end_time字段。 Then my
dosomething
method would look something like this: 然后我的
dosomething
方法看起来像这样:
def self.dosomething(model_id)
model = Model.find(model_id)
begin
model.start!
# do some long work ...
rescue Exception => e
# ...
ensure
model.finish!
end
end
The start! 开始! and finish!
并完成! methods just record the current time and save the model.
方法只记录当前时间并保存模型。 Then I would have a
completed?
然后我会
completed?
一个completed?
method that your AJAX can poll to see if the job is finished. 您的AJAX可以轮询以查看作业是否已完成的方法。
def completed?
return true if start_time and end_time
return false
end
There are many ways to do this but I find this method simple and works well for me. 有很多方法可以做到这一点,但我发现这种方法很简单,对我来说效果很好。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.