简体   繁体   English

kafka流处理器Api反序列化avro记录

[英]kafka streams processor Api to deserialize a avro record

I am streaming the real time data from Kafka. 我正在从Kafka流式传输实时数据。 But the data is in Avro format. 但是数据为Avro格式。 Unable to deserialize as Json. 无法反序列化为Json。 Iam using Kafka Stream Low level Processor API. Iam使用Kafka Stream低级处理器API。 How to deserialize Avro record? 如何反序列化Avro记录?

def orderStreamData(builder: KStreamBuilder, inTopic: String, outTopic: String): TopologyBuilder = {
    builder
      .addSource("source1", stringDe, stringDe, inTopic)//adding source topic
      //now adding processor class using ProcessSupplier
      .addProcessor("order", new ProcessorSupplier[String, String] {
      override def get(): Processor[String, String] = new ProcessorImpl
    }, "source1")
      //adding local state store for stateful operations
      .addStateStore(Stores.create("tester").withStringKeys.withStringValues.inMemory.build, "order")
      //adding destination topic for the processed data to go
      .addSink("sink", outTopic, stringSer, stringSer, "order")
  }

class ProcessorImpl extends AbstractProcessor[String, String]{

  var keyValueStore: KeyValueStore[String, String] = _
  var processorContext: ProcessorContext = _

  override def init(context: ProcessorContext): Unit = {
    processorContext = context
    processorContext.schedule(10000L)
    keyValueStore = processorContext.getStateStore("tester").asInstanceOf[KeyValueStore[String, String]]
    Objects.requireNonNull(keyValueStore, "State Store can't be null")
  }
/**
  * here logic is implemented
  * every value for a key must be greater than the previous value
  * */
  override def process(key: String, value: String): Unit = {
    //accessing local state store for last value saved for this key

  }
}
@Override
  public T deserialize(String inTopic, byte[] data) {
    try {
      T result = null; // Intialize result to null
  if (data != null) {

    DatumReader<GenericRecord> datumReader =
        new SpecificDatumReader<>(targetType.newInstance().getSchema());
    Decoder decoder = DecoderFactory.get().binaryDecoder(data, null);

    result = (T) datumReader.read(null, decoder);
    LOGGER.debug("deserialized data='{}'", result);
  }
  return result;
} catch (Exception ex) {
  throw new SerializationException("can not deserialize"+data+"exception"+ex);
}

} }

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM