[英]How does Celery actually execute the task?
I have a text file inside directory the_files
that contains several line like this 我在目录
the_files
中有一个文本文件,其中包含几行这样的内容
aaaaabbbb cccc--ddddeee ffff
gggjjjkkk eers--kklliii kkll
...
I wrote a celery script that manipulate each line in the text file. 我写了一个芹菜脚本来操纵文本文件中的每一行。
from celery import Celery
import os
app = Celery('tasks', broker='amqp://guest@localhost//')
path = "the_files/"
@app.task
def do_task_txt():
dir_path = os.listdir(path)
for file in dir_path:
if file.endswith(".txt"):
f = open(path + file, "r")
for line in f:
string1 = line[0:14].replace(" ", "")
string2 = line[16:].replace(" ", "")
#print string1, string2
return string1, string2
f.close()
When I run this script with celery it provides the following result 当我用celery运行此脚本时,它提供以下结果
[tasks]
. tasks.do_task_txt
[2017-03-22 13:51:00,713: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2017-03-22 13:51:00,791: INFO/MainProcess] mingle: searching for neighbors
[2017-03-22 13:51:01,966: INFO/MainProcess] mingle: all alone
[2017-03-22 13:51:02,055: INFO/MainProcess] celery@Ling-Air ready.
[2017-03-22 13:51:25,624: INFO/MainProcess] Received task: tasks.do_task_txt[77166520-21a8-466e-9522-cb2b1821a185]
[2017-03-22 13:51:26,152: INFO/PoolWorker-2] Task tasks.do_task_txt[77166520-21a8-466e-9522-cb2b1821a185] succeeded in 0.00866508999752s: ('aaaaabbbbcccc', 'ddddeeeffff')
It showed the first line only. 它仅显示第一行。
I was hoping to get it to show for every line perhaps like this maybe? 我希望能像这样显示在每一行中?
[2017-03-22 13:51:26,152: INFO/PoolWorker-2] Task tasks.do_task_txt[77166520-21a8-466e-9522-cb2b1821a185] succeeded in 0.00866508999752s: ('aaaaabbbbcccc', 'ddddeeeffff'),('gggjjjkkkeers', 'kklliiikkll'),(.....,...)
I checked my script by calling print string1, string2
and it did print the result as what I expected like this 我通过调用
print string1, string2
检查了我的脚本,它确实按照我期望的那样打印了结果
aaaaabbbbcccc ddddeeeffff
gggjjjkkkeers kklliiikkll
...
My question is how does Celery execute the task? 我的问题是芹菜如何执行任务? When I execute the task
do_task_txt
, it shows only one line from the file that has been manipulated. 当我执行任务
do_task_txt
,它仅显示已处理文件中的一行。 How do I show all of the line that has been manipulated instead of one line only? 如何显示所有已操纵的行而不是仅显示一行?
Thank you for your suggestion. 谢谢你的建议。
first f.close()
is never called when inside the if-statement, so file handle will remain. 在if语句中时,永远不会调用第一个
f.close()
,因此文件句柄将保留。
Second your task returns a tuple which celery (or the broker) do not parse correctly. 其次,您的任务返回一个元组,芹菜(或经纪人)无法正确解析。
Return a merged string instead: 而是返回一个合并的字符串:
return ", ".join(string1, string2)
When you return
, the function execution is finished, it won't process left lines and other files. return
,函数执行完成,不会处理左行和其他文件。 You should save results and return finally. 您应该保存结果并最终返回。
@app.task
def do_task_txt():
dir_path = os.listdir(path)
result = []
for file in dir_path:
if file.endswith(".txt"):
f = open(path + file, "r")
for line in f:
string1 = line[0:14].replace(" ", "")
string2 = line[16:].replace(" ", "")
#print string1, string2
result += [string1, string2]
f.close()
return result
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.