Confluent Kafka：如何在 confluent-kafka-python 客户端中指定序列化和分区？

Question

1) In Java Client, using producer.put() we can add serializers for Integers and Strings. How do we do this for confluent-kafka-python client? 2) How do we serialize protoBuf records?

Answer 1

In recent version i guess it gives No such configuration property: "value.serializer" thus we will need to user SerializingProducer and value.serializer value should be callback

from confluent_kafka import SerializingProducer
import json


def json_serializer(msg, s_obj):
    return json.dumps(msg).encode('utf-8')


conf = {'bootstrap.servers': 'localhost:29092',
        'value.serializer': json_serializer}

producer_client = SerializingProducer(conf)


def producer_json_msg(topic_name, **args):
    producer_client.produce(topic_name, **args)

producer_json_msg('test_two', value={'test': "this is fine"})
producer_client.flush()

Answer 2

For question #1: Add key.serializer and value.serializer in the config, as show below:

from confluent_kafka import Producer
import socket
conf = {'bootstrap.servers': "host1:9092,host2:9092",
    'client.id': socket.gethostname(),
    'key.serializer': 'key serializer classpath',
    'value.serializer': 'value serializer classpath',
    'default.topic.config': {'acks': 'all'}}
producer = Producer(conf)

For question #2: Implement your own ProtobufSerializer class and set {key|value}.serializer to it.