[英]NullPointerException exception when using Flink's leftOuterJoinLateral in Scala
I am trying to follow the documentation and create a Table Function to "flatten" some data.我正在尝试遵循文档并创建一个表 Function 来“展平”一些数据。 The Table Function seems to work fine when using the
joinLateral
to do the flattening.使用
joinLateral
进行展平时,表 Function 似乎工作正常。 When using leftOuterJoinLateral
though, I get the following error.但是,当使用
leftOuterJoinLateral
时,我收到以下错误。 I'm using Scala and have tried both Table API and SQL with the same result:我正在使用 Scala 并尝试了表 API 和 SQL ,结果相同:
Caused by: java.lang.NullPointerException: Null result cannot be stored in a Case Class.
原因:java.lang.NullPointerException:Null 结果不能存储在案例 Class 中。
Here is my job:这是我的工作:
import org.apache.flink.streaming.api.scala.StreamExecutionEnvironment
import org.apache.flink.table.api.scala.StreamTableEnvironment
import org.apache.flink.table.api.scala._
import org.apache.flink.streaming.api.scala._
import org.apache.flink.table.functions.TableFunction
object example_job{
// Split the List[Int] into multiple rows
class Split() extends TableFunction[Int] {
def eval(nums: List[Int]): Unit = {
nums.foreach(x =>
if(x != 3) {
collect(x)
})
}
}
def main(args: Array[String]): Unit = {
val env = StreamExecutionEnvironment.createLocalEnvironment()
val tableEnv = StreamTableEnvironment.create(env)
val splitMe = new Split()
// Create some dummy data
val events: DataStream[(String, List[Int])] = env.fromElements(("simon", List(1,2,3)), ("jessica", List(3)))
val table = tableEnv.fromDataStream(events, 'name, 'numbers)
.leftOuterJoinLateral(splitMe('numbers) as 'number)
.select('name, 'number)
table.toAppendStream[(String, Int)].print()
env.execute("Flink jira ticket example")
}
}
When I change .leftOuterJoinLateral
to .joinLateral
I get the expected result:当我将
.leftOuterJoinLateral
更改为.joinLateral
时,我得到了预期的结果:
(simon,1)
(simon,2)
When using the .leftOuterJoinLateral
I would expect something like:使用
.leftOuterJoinLateral
时,我会期望类似:
(simon,1)
(simon,2)
(simon,null) // or (simon, None)
(jessica,null) // or (jessica, None)
Seems like this might be a bug with the Scala API?似乎这可能是 Scala API 的错误? I wanted to check here first before raising a ticket just in case I'm doing something stupid!
我想在提出罚单之前先检查这里,以防我做一些愚蠢的事情!
The problem is that Flink per default does expect that all fields of a row are non-null.问题是 Flink 默认情况下确实期望一行的所有字段都是非空的。 That's why the program fails when it sees the
null
result from the outer join operation.这就是程序在看到外连接操作的
null
结果时失败的原因。 In order to accept null
values, you either need to disable the null check via为了接受
null
值,您需要通过以下方式禁用 null 检查
val tableConfig = tableEnv.getConfig
tableConfig.setNullCheck(false)
Or you must specify the result type to tolerate null values, eg specifying a custom POJO output type:或者您必须指定结果类型以容忍 null 值,例如指定自定义 POJO output 类型:
table.toAppendStream[MyOutput].print()
with和
class MyOutput(var name: String, var number: Integer) {
def this() {
this(null, null)
}
override def toString: String = s"($name, $number)"
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.