Flink Window TVF和CEP SQL的相關規則是什么？

Question

我正在嘗試解析 Flink 窗口 TVF sql 列級沿襲，我初始化了一個自定義 FlinkChainedProgram 並設置了一些 Opt 規則。

除 Window TVF SQL 和 CEP SQL 外，大部分工作正常。

例如，我得到一個合乎邏輯的計划

insert into sink_table(f1, f2, f3, f4) 
       SELECT cast(window_start as String),
              cast(window_start as String), 
              user_id, 
              cast(SUM(price) as Bigint) 
       FROM TABLE(TUMBLE(TABLE source_table, DESCRIPTOR(event_time), INTERVAL '10' MINUTES))
       GROUP BY window_start, window_end, GROUPING SETS ((user_id), ());

rel#1032:FlinkLogicalCalc.LOGICAL.any.None: 0.[NONE].[NONE](input=FlinkLogicalAggregate#1030,select=CAST(window_start) AS EXPR$0, CAST(window_start) AS EXPR$1, null:BIGINT AS EXPR$2, user_id, null:VARCHAR(2147483647) CHARACTER SET "UTF-16LE" AS EXPR$4, CAST($f4) AS EXPR$5)

如我們所見，優化的 RelNode 樹包含 null 列，因此 MetadataQuery 無法獲取原始列信息。

我應該在邏輯優化階段設置什么規則來解析 Window TVF SQL 和 CEP SQL？ 謝謝

Answer 1

解決了Flink CEP SQL的字段血緣關系方法，在org.apache.calcite.rel.metadata.org.apache.calcite.rel.metadata中添加了getColumnOrigins(Match rel, RelMetadataQuery mq, int iOutputColumn)方法。 相關列來源

/**
 * Support field blood relationship of CEP.
 * The first column is the field after PARTITION BY, and the other columns come from the measures in Match
 */
public Set<RelColumnOrigin> getColumnOrigins(Match rel, RelMetadataQuery mq, int iOutputColumn) {
    if (iOutputColumn == 0) {
        return mq.getColumnOrigins(rel.getInput(), iOutputColumn);
    }
    final RelNode input = rel.getInput();
    RexNode rexNode = rel.getMeasures().values().asList().get(iOutputColumn - 1);

    RexPatternFieldRef rexPatternFieldRef = searchRexPatternFieldRef(rexNode);
    if (rexPatternFieldRef != null) {
        return mq.getColumnOrigins(input, rexPatternFieldRef.getIndex());
    }
    return null;
}

private RexPatternFieldRef searchRexPatternFieldRef(RexNode rexNode) {
    if (rexNode instanceof RexCall) {
        RexNode operand = ((RexCall) rexNode).getOperands().get(0);
        if (operand instanceof RexPatternFieldRef) {
            return (RexPatternFieldRef) operand;
        } else {
            // recursive search
            return searchRexPatternFieldRef(operand);
        }
    }
    return null;
}

源碼地址： https://github.com/HamaWhiteGG/flink-sql-lineage/blob/main/src/main/java/org/apache/calcite/rel/metadata/RelMdColumnOrigins.java

我已經給出了詳細的測試用例，你可以參考： https://github.com/HamaWhiteGG/flink-sql-lineage/blob/main/src/test/java/com/dtwave/flink/lineage/cep/CepTest。 java

flink cep sql 測試用例