简体   繁体   中英

Is it possible to modify an apache beam stream with pub/sub input without interruptions?

1) I wrote a pipeline which has as an input pub/sub stream. I want to modify it with --update or --jobName option. What it's happening to the data on pub/sub and in the whole pipeline?

2) I must turn off a pipeline for some reason for: a) for a few hours or b) for a few days. Is it possible to replay this data in some easy way to the restarted pipeline so everything is processed as before in a stream mode. I've heard that you can replay pub/sub for 7 days.

1) If you update a streaming pipeline, the pipeline running now will stop pulling data from pubsub, and saves intermediate window state before the new pipeline is launched. So there will be no data loss or duplication.

2) Currently Dataflow does not support resumed pipeline execution.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM