Am using cloudScheduler to schedule the dataflow job. Where i need to pass the rundate/runtime/current date dynamically to dataflow job using cloudsheduler body. Am reading the body parameters using Java Options in Dataflow job code.
Looking for syntax to read header X-CloudScheduler-ScheduleTime in cloudschduler body
body = base64encode(<<-EOT { "jobName": "${each.value.name}_${http_target.h}", "parameters": { "bigtableInstanceId": "${var.cloudscheduler_bigtable_instanceid}", "bigtableProjectId": "${var.project_id}", "bigtableTableId": "${each.value.bigtableTableId}", "inputFilePattern": "${each.value.inputFilePattern}", "runningdate":????????
"region": "${var.cloudscheduler_region}", }, "environment": { "numWorkers": "2", "tempLocation": "gs://gcslocation/temp2", "zone": "${var.cloudscheduler_region}-a" } } EOT ) }
How to pass the scheduled date/rundate in the body "runningdate":???????? of cloudscheduler which we read in dataflow job code.
(Simple Approach) Try using Cloud Functions rather than Cloud Scheduler .
Cloud Scheduler is supposed work like a unix cron job, to schedule and run jobs.
Cloud Functions are mainly useful for uses cases where arguments need to be parsed & dynamic decision to be made like should I run Dataflow job or any as such.
If you have requirement/such use case to use Cloud Scheduler
, then what you can do is
If I misunderstood, your question, please do elaborate with more details & examples.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.