简体   繁体   English

KSQL:UDF 不接受参数 (STRING, STRING)

[英]KSQL: UDF does not accept parameters (STRING, STRING)

I'm running in to a problem with KSQL while trying to set up an ETL pipeline using a UDF.我在尝试使用 UDF 设置 ETL 管道时遇到了 KSQL 问题。 At some point in the ETL process I need to isolate specific info from a description field (VARCHAR) in my data.在 ETL 过程的某个时刻,我需要从数据中的描述字段 (VARCHAR) 中分离出特定信息。 A made-up example for context:上下文的一个虚构示例:

description = "species=dog.sex=male.color=blonde.age=10." description = "物种=dog.sex=male.color=blonde.age=10。" (the real data is formatted in the same way) (真实数据格式相同)

I've written a simple UDF to isolate any information on demand.我编写了一个简单的 UDF 来按需隔离任何信息。 It looks like this:它看起来像这样:

package com.my.package;

/** IMPORTS **/
import io.confluent.ksql.function.udf.Udf;
import io.confluent.ksql.function.udf.UdfDescription;

/** ClASS DEFINITION **/
@UdfDescription(name = "extract_from_description",
                author = "Me",
                version = "0.0.1",
                description = "Given a description and a request for information, isolates and returns the requested information. Pass requested tag as 'tag='".) 
public class Extract_From_Description {

    @Udf(description = "Given a description and a request for information, isolates and returns the requested information. Pass requested tag as 'tag='.)
    public String extract_from_description(final String description, final String request) {
        return description.split(request)[1].split("\\.")[0];
    }
}

I can upload and register the function just fine, it's listed and described properly when I run:我可以很好地上传和注册该函数,它在我运行时被正确列出和描述:

ksql> list functions;
ksql> describe function EXTRACT_FROM_DESCRIPTION;

I call the function like this to create a new stream:我调用这样的函数来创建一个新的流:

CREATE STREAM result AS
    SELECT recordId,
           OtherVariables,
           EXTRACT_FROM_DESCRIPTION(description, 'species=') AS species
    FROM parent_stream
    EMIT CHANGES;

There I get an error I can't make sense of:在那里我得到一个我无法理解的错误:

Function 'extract_from_description' does not accept parameters (STRING, STRING).函数“extract_from_description”不接受参数(STRING、STRING)。 Valid alternatives are:有效的替代方案是:

Apparently KSQL can't properly interpret what the input for the function is supposed to be (looks like it expects no input?) and I can't figure out why.显然,KSQL 无法正确解释函数的输入应该是什么(看起来它不需要输入?),我不知道为什么。 I've read through documentation to see if I define my function in a weird way but can't find any differences between the examples and my function.我通读了文档,看看我是否以一种奇怪的方式定义了我的函数,但在示例和我的函数之间找不到任何差异。 I did notice there are supposed to be several ways to define the input a function takes and tried them all, but the result is always the same.我确实注意到应该有几种方法来定义函数接受的输入并尝试了所有方法,但结果总是相同的。

I use Maven to create the jar file for this function (JDK1.8.0_201).我使用 Maven 为这个函数 (JDK1.8.0_201) 创建 jar 文件。 Can anyone help me figure out what's going on?谁能帮我弄清楚发生了什么?

TL;DR: My KSQL UDF doesn't accept input of type (String, String) even though the function specifies the input should be of type (String, String) TL;DR:我的 KSQL UDF 不接受 (String, String) 类型的输入,即使函数指定输入应该是 (String, String) 类型

Found the problem, answering here for anyone that might run in to the same problem.发现问题,在这里为可能遇到相同问题的任何人回答。 You need to specify the parameters using @UdfParameter, like this:您需要使用@UdfParameter 指定参数,如下所示:

import io.confluent.ksql.function.udf.UdfParameter; // add this to the list of imports

// add @UdfParameter(name) to each input variable
public String extract_from_description(@UdfParameter(value = "description") final String description, @UdfParameter(value = "request") final String request){
           
  function body

}

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM