在 apache beam 中创建自定义接收器

Question

I am using apache beam and trying to Create a custom sink, unfortunately cannot find any guides on how to create a custom sink.Can someone guide.

Previously in Dataflow i used to override the Sink available in com.google.cloud.dataflow.sdk.io.Sink

I cannot seem to find a similar calls in Beam.Is it still avialble in beam somewhere?

I am using beam 2.3 sdk and Java

Answer 1

Please see https://beam.apache.org/documentation/io/authoring-overview/ and see implementation of the other write() transforms included with the Beam SDK.

The Sink interface has been deleted because we have found that there are no cases where it provides any value over just implementing a composite transform made up of ParDo, GBK and other transforms. Please also see this recent talk of mine ( slides ) about IO in Beam and about why sources/sinks should not be a special API, but should be simply composite transforms.