简体   繁体   English

如何在 Argo 工作流程中增加计数器?

[英]how to increment a counter in Argo workflow?

I have written a argo dag to trigger spark job in recursion until the condition satisfies.我写了一个 argo dag 以递归方式触发 spark 作业,直到条件满足。 I have a counter parameter which needs to be incremented by 1 after every successful completion of spark job.我有一个计数器参数,每次成功完成 spark 作业后都需要增加 1。 But this isnt happening.但这并没有发生。 Here is the snippet of my workflow.这是我的工作流程的片段。

  templates:
    - name: test-dag
      dag:
        tasks:
        - name: test-spark-job
          template: test-spark-job
        - name: loop-it
          template: backfill-dag
          dependencies: [backfill-spark-job]
          when: "{{=asInt(workflow.parameters.counter)}} <= {{=asInt(workflow.parameters.batchsize)}}"
          arguments:
            parameters:
              - name: counter
                value: {{=asInt(workflow.parameters.counter)}}+1

It +1 should be part of the expression.+1应该是表达式的一部分。 Try:尝试:

arguments:
  parameters:
    - name: value
      value: "{{=asInt(workflow.parameters.counter) + 1}}"

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 使用 Argo 工作流的 K8S Spark - K8S Spark using Argo Workflow Apache Spark:如何创建计数器 - Apache Spark: how to create a counter 如何将 spark 参数传递给 dataproc 工作流模板? - How to pass spark parameter to a dataproc workflow template? 在不使用 Oozie、Airflow 等工作流管理器的情况下,如何在 Apache Spark 中执行工作流的顺序/并行任务? - How do I execute sequential/parallel tasks of a workflow in Apache Spark without the use of workflow managers such as Oozie, Airflow? 如何在spark SQL中实现自动增量(PySpark) - How to implement auto increment in spark SQL(PySpark) 如何从kafka流到cassandra并增加计数器 - how to stream from kafka to cassandra and increment counters 如果列的值不在 Pyspark 中,如何将值加一 - How to increment the value by one if the value of the column is not there in Pyspark 如何为 oozie 工作流中的不同操作指定不同的用户库路径 - How to specify different user library path for different actions in an oozie workflow Spark Scala:如何同时过滤RDD和更新计数器 - Spark Scala: How to filter RDD and update counter at the same time 如何基于固定在Window中的列值增加计数器? - How to increment counters based on a column value being fixed in a Window?
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM