[英]Apache Flink: can't use writeAsCsv() with a datastream of subclass tuple
As recommended here: Best Practices - Naming large TupleX types . 如此处推荐: 最佳实践-命名大型TupleX类型 。 I'm using a POJO instead of a Tuple for my data stream.
我在数据流中使用的是POJO而不是元组。
This is how defined my POJO: 这就是我的POJO的定义方式:
public class PositionEvent extends Tuple8<Integer, String, Integer,
Integer, Integer, Integer, Integer, Integer>
If I try to save a data stream of PositionEvent
to a csv file, an exception is thrown: 如果我尝试将
PositionEvent
的数据流保存到csv文件中,则会引发异常:
source.filter((PositionEvent e) -> e.speed > MAXIMUM_SPEED)
.writeAsCsv(String.format("%s/%s", outputFolder, SPEED_RADAR_FILE))
Exception in thread "main" java.lang.IllegalArgumentException: The writeAsCsv() method can only be used on data streams of tuples.
线程“主”中的异常java.lang.IllegalArgumentException:writeAsCsv()方法只能在元组的数据流上使用。
However, if I explicitly cast PositionEvent
to Tuple8, it works: 但是,如果我将
PositionEvent
明确地强制转换为Tuple8,它将起作用:
source.filter((PositionEvent e) -> e.speed > MAXIMUM_SPEED)
.map((PositionEvent e) ->
(Tuple8<Integer, String, Integer, Integer,
Integer, Integer, Integer, Integer>) e)
.writeAsCsv(String.format("%s/%s", outputFolder, SPEED_RADAR_FILE))
Should not Flink detect that the objects in the data stream are of the Tuple
subclass? Flink是否不应检测到数据流中的对象属于
Tuple
子类?
==================== ====================
Edit: (thanks to twalthr) 编辑:(感谢twalthr)
Ok, this is my POJO now: 好的,这是我现在的POJO:
import org.apache.flink.api.java.tuple.Tuple8;
public class PositionEvent extends Tuple8<Integer, String, Integer,
Integer, Integer, Integer, Integer, Integer> {
public PositionEvent() {
}
public PositionEvent(int timestamp, String vid, int speed, int xway,
int lane, int dir, int seg, int pos) {
super(timestamp, vid, speed, xway, lane, dir, seg, pos);
}
public int getSpeed() {
return f2;
}
}
This was my POJO before: 这是我以前的POJO:
public class PositionEvent extends Tuple8<Integer, String, Integer,
Integer, Integer, Integer, Integer, Integer> {
public int timestamp;
public String vid;
public int speed;
public int xway;
public int lane;
public int dir;
public int seg;
public int pos;
public PositionEvent() {
}
public PositionEvent(int timestamp, String vid, int speed, int xway,
int lane, int dir, int seg, int pos) {
super(timestamp, vid, speed, xway, lane, dir, seg, pos);
}
}
Now I don't need to explicitly cast my POJO. 现在,我不需要显式转换POJO。
It seems that you not only extended Tuple8
but also added additional fields like e.speed
. 看来您不仅扩展了
Tuple8
而且还添加了其他字段,例如e.speed
。 This implicitly makes your type a POJO. 这隐式地使您的类型成为POJO。 For naming your fields and remain a efficient tuple type, you can simply implement a getter but don't add additional fields.
为了命名字段并保持有效的元组类型,您可以简单地实现getter而不添加其他字段。 Otherwise you can simply use a POJO instead of tuple.
否则,您可以简单地使用POJO而不是元组。
It might also be worth to look into Flink's Table & SQL API . 也许还值得研究Flink的Table&SQL API 。 It aims to ease the development by handling all types automatically.
它旨在通过自动处理所有类型来简化开发。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.