[英]How to handle a task failure in prefect and return SUCCESS with the on_failure param?
我有一個prefect
的Flow
,其task
的輸出是dataframe
。 在下面提供的示例中,它總是失敗。 我希望該task
使用@task(on_failure=handle_task_fail)
返回一個狀態為SUCCESS
的空dataframe
。 實現這一目標的正確語法是什么?
from pprint import pprint
import pandas as pd
from prefect import Flow, task
from prefect.engine.signals import SUCCESS
def handle_disambig_error(task, old_state, new_state):
if new_state.is_failed():
new_state.result["wiki_df"] = pd.DataFrame()
# Is this needed?
#set state to SUCCESS
return new_state
@task(on_failure=handle_disambig_error)
def get_wiki_resource():
wiki_df = pd.DataFrame(
{
"a":[1],
"b":[1/0]
}
)
return wiki_df
with Flow("Always Fail") as flow:
wiki_df = get_wiki_resource()
state = flow.run()
task_state = state.result[wiki_df]
pprint(task_state.result)
追溯:
Traceback (most recent call last):
File "/miniconda3/lib/python3.7/site-packages/prefect/engine/runner.py", line 161, in handle_state_change
new_state = self.call_runner_target_handlers(old_state, new_state)
File "/miniconda3/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 120, in call_runner_target_handlers
new_state = handler(self.task, old_state, new_state) or new_state
File "/miniconda3/lib/python3.7/site-packages/prefect/utilities/notifications.py", line 69, in state_handler
fn(obj, new_state)
TypeError: handle_disambig_error() missing 1 required positional argument: 'new_state'
[2020-01-28 17:39:41,759] INFO - prefect.TaskRunner | Task 'get_wiki_resource': finished task run for task with final state: 'Failed'
[2020-01-28 17:39:41,762] INFO - prefect.FlowRunner | Flow run FAILED: some reference tasks failed.
有些地方我搜索了State Handlers , Logging with a State Handler
這里發生了兩件事:
1.) 通用狀態處理程序:這些可以通過state_handlers
kwarg 設置,並且將在每次狀態更改時調用。 狀態處理程序需要具有簽名state_handler(task: Task, old_state: State, new_state: State) -> Optional[State]
(這是您正在使用的簽名); 調用此處理程序后任務的狀態將是從處理程序返回的狀態,如果返回None
則為new_state
。
2.) On Failure 回調:您在這里使用的on_failure
kwarg 旨在成為狀態處理程序的便捷 API; 傳遞給這個關鍵字的函數需要有簽名fn(task: Task, state: State) -> None
並且只有當這個 Task 進入Failed
狀態時才會被調用。 請注意,失敗回調不能像狀態處理程序那樣改變任務的狀態。
在您的示例中,您似乎混合了兩個關鍵字參數。 我相信下面的代碼會做你所期望的:
from prefect.engine.state import Success
def handle_disambig_error(task, old_state, new_state):
if new_state.is_failed():
return_state = Success(result=pd.DataFrame())
else:
return_state = new_state
return return_state
@task(state_handlers=[handle_disambig_error])
def get_wiki_resource():
return df
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.