MetaFlow permits you to set the maximum number of concurrent tasks using the --max-workers
CLI flag (ref: https://docs.metaflow.org/metaflow/scaling#safeguard-flags ). However, I would like to avoid setting this every time.
Is it possible to set the --max-workers
flag from the Python definition of the FlowSpec (without the CLI)?
The CLI flag sets the METAFLOW_MAX_WORKERS
environment variable. You can set this within Python by setting this environment variable before you define your FlowSpec:
# force MetaFlow to use only 1 concurrent task during a foreach fanout
os.environ.update({"METAFLOW_MAX_WORKERS": "1"})
Source: chat with @Savin
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.