将参数化的 Prefect 流程与因参数而异的计划相结合

Question

I'd like to be be able to re-use the same Parameterized Prefect flow, but where the Schedule varies by inputs. So for example:

from prefect import task, Flow, Task, Parameter
from prefect.schedules import CronSchedule

diurnal   = ['rooster', 'dog']
nocturnal = ['owl', 'hampster']

# Schedules
diurnal_schedule   = CronSchedule("50 7 * * mon,wed")
nocturnal_schedule = CronSchedule("15 12 * * tue,thu")

# Flow is common to both types, though with different schedules.
with Flow(name="wakuptime") as this_flow:
    animals         = Parameter("animals")
    wakeup(animals)

this_flow.run(parameters = dict(animals = diurnal)) on diurnal_schedule
this_flow.run(parameters = dict(animals = nocturnal)) on nocturnal_schedule

Any suggestions?

Answer 1

This is now possible on the master branch of Prefect and will be released with 0.9.2 next week: the Clocks API now allows for providing parameters to each clock that will be passed to each flow run generated from that clock. In your case, you can do:

from prefect import task, Flow, Task, Parameter
from prefect.schedules import clocks, Schedule

diurnal   = ['rooster', 'dog']
nocturnal = ['owl', 'hampster']

# Clocks
diurnal_clock   = clocks.CronClock("50 7 * * mon,wed", parameter_defaults={"animals": diurnal})
nocturnal_clock = clocks.CronClock("15 12 * * tue,thu", parameter_defaults={"animals": nocturnal})

# the full schedule
schedule = Schedule(clocks=[diurnal_clock, nocturnal_clock])

# Flow is common to both types, though with different schedules.
with Flow(name="wakuptime", schedule=schedule) as this_flow:
    animals = Parameter("animals", default=[])
    wakeup(animals)

# will run on the schedule with varying parameter values
this_flow.run()