[英]Apache beam Dataflow SDK error with example
I'm trying one of the beam google dataflow pipeline examples, but i'm bumping into a exception regarding MapElements and methods SingleFunction / SerializableFunction calls. 我正在尝试谷歌数据流管道示例之一,但我碰到了有关MapElements和方法SingleFunction / SerializableFunction调用的异常。 The code snippet is the following:
代码段如下:
static class ParseTableRowJson extends SimpleFunction<String, TableRow> {
@Override
public TableRow apply(String input) {
try {
return Transport.getJsonFactory().fromString(input, TableRow.class);
} catch (IOException e) {
throw new RuntimeException("Failed parsing table row json", e);
}
}
}
......
p.apply(TextIO.read().from(options.getInput()))
.apply(MapElements.via(new ParseTableRowJson()))
.apply(new ComputeTopSessions(samplingThreshold))
.apply("Write",
TextIO.write().withoutSharding().to(options.getOutput()));
The exception in that its an ambiguous call to the methods: 例外之处在于它对方法的模棱两可:
Ambiguous method call. Both
via (SimpleFunction<String, TableRow>) in MapElements and
via (SerializableFunction) in MapElements match
Has someone else bumped into the same exception and got a way around it? 是否有人碰到了相同的异常并有办法解决?
The full example is in github ( https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/complete/TopWikipediaSessions.java ). 完整的示例位于github( https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/complete/TopWikipediaSessions.java )中。
Thanks, 谢谢,
Fernando 费尔南多
This seems to have been fixed in the code at HEAD. 这似乎已在HEAD的代码中修复。 Specifically,
MapElements
no longer has two static versions of via
. 具体来说,
MapElements
不再具有via
两个静态版本。 Short-term, you can either install Beam from HEAD or update the example to use ParDo
directly by making the ParseTableRowJson
a DoFn
instead of a SimpleFunction
. 短期,你可以从头梁安装或更新的例子使用
ParDo
直接通过使ParseTableRowJson
一个DoFn
而不是SimpleFunction
。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.