[英]How to produce a json object message into kafka topic using java(spring)?
I want to produce a message into kafka topic. 我想向kafka主题发送消息。 That message should have this pattern:
该消息应具有以下模式:
{"targetFileInfo":{"path":"2018-05-07-10/row01-small-01.txt.ready"}}
I know that is a json pattern, so how can i convert that json in String? 我知道这是一个json模式,那么如何在String中转换该json?
I use a maven project, so which dependencies are needed to use 我使用一个Maven项目,因此需要使用哪些依赖项
String stringData = JSON.stringify({"targetFileInfo":{"path":"2018-05-07-10/row01-small-01.txt.ready"}});
So I think it is better don't convert Json to string and send indeed that massage into kafka topic. 因此,我认为最好不要将Json转换为字符串,并且不要将按摩发送到kafka主题中。
My Code is like that, it can send a String but i don't know how i can modify my code to send the massage above. 我的代码就是这样,它可以发送一个字符串,但是我不知道如何修改我的代码以发送上述消息。 maybe you can help me.
也许你可以帮我。
Producer<String, String> producer = null;
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("acks", "all");
props.put("retries", 0);
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
producer = new KafkaProducer<>(props);
String msg = "welcome";
producer.send(new ProducerRecord<String, String>("event", msg));
producer.close();
As per the comment you need to send JsonNode
as message on kafka. 根据评论,您需要在kafka
JsonNode
作为消息发送。 Write a custom Serializer / Deserializer for the same. 为此编写一个自定义的序列化器/反序列化器。
import java.io.IOException;
import java.util.Map;
import org.apache.kafka.common.serialization.Deserializer;
import org.apache.kafka.common.serialization.Serializer;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
public class JsonNodeSerDes implements Serializer<JsonNode>, Deserializer<JsonNode> {
private ObjectMapper mapper = new ObjectMapper();
@Override
public byte[] serialize(String topic, JsonNode data) {
try {
return mapper.writeValueAsBytes(data);
} catch (JsonProcessingException e) {
return new byte[0];
}
}
@Override
public JsonNode deserialize(String topic, byte[] data) {
try {
return mapper.readValue(data, JsonNode.class);
} catch (IOException e) {
return null;
}
}
@Override
public void configure(Map<String, ?> configs, boolean isKey) {
}
@Override
public void close() {
}
}
I wrote serializer / deserializer in the same class. 我在同一个类中编写了序列化器/反序列化器。 You can separate them in two class (one implementing
Serializer
, another implementing Deserializer
). 您可以将它们分为两个类(一个实现
Serializer
,另一个实现Deserializer
)。
While creating KafkaProducer
you need to provide "value.serializer" config and "value.deserializer" config for KafkaConsumer
. 在创建
KafkaProducer
您需要提供“value.serializer”配置和“value.deserializer”配置KafkaConsumer
。
External Dependencies used: 使用的外部依赖项:
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.8.8</version>
</dependency>
That solved my problem: 那解决了我的问题:
Producer<String, String> producer = null;
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("acks", "all");
props.put("retries", 0);
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
producer = new KafkaProducer<>(props);
try {
producer = new KafkaProducer<String, String>(props);
} catch (Exception e) {
e.printStackTrace();
}
blobStorageChecker = new BlobStorageChecker();
String folder = blobStorageChecker.getCurrentDateUTC();
String msg = "{\"targetFileInfo\":{\"path\":\"test/"+folder+"row01-small.txt\"},\"sourceFileInfo\":{\"lastModifiedTime\":1525437960000,\"file\":\"/row01-small-01.txt\",\"filename\":\"/data/row01/row01-small.txt\",\"size\":19728,\"remoteUri\":\"ftp://waws-prod-am2-191.ftp.net/data/orsted-real/inbound/row01\",\"contentEncoding\":\"\",\"contentType\":\"\"}}";
ProducerRecord<String, String> record = new ProducerRecord<String, String>("event-orsted-v1", null, msg);
if (producer != null) {
try {
Future<RecordMetadata> future = producer.send(record);
RecordMetadata metadata = future.get();
} catch (Exception e) {
System.err.println(e.getMessage());
e.printStackTrace();
}
}
producer.close();
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.