如何使用Python confluent_kafka admin NewTopic *args 或 **kwargs 微调主题配置

Question

How to use Python confluent_kafka admin NewTopic *args or **kwargs to fine tune topic configuration?

This is the dock-string I found:

confluent_kafka.cimpl.NewTopic def __init__(self,
         topic: str,
         num_partitions: int,
         replication_factor: int = None,
         *args: Any,
         **kwargs: Any) -> None
NewTopic specifies per-topic settings for passing to passed to AdminClient.create_topics().
Params:
topic – Topic name
num_partitions – Number of partitions to create
replication_factor – Replication factor of partitions, or -1 if replica_assignment is used.

In the source code I found c file code and documentation:

typedef struct {
    PyObject_HEAD
    char *topic;
    int   num_partitions;
    int   replication_factor;
    PyObject *replica_assignment;  /**< list<int> */
    PyObject *config;              /**< dict<str,str> */
} NewTopic;

docstring:

    "  :param dict config: Dict (str:str) of topic configuration. See http://kafka.apache.org/documentation.html#topicconfigs\n"
    "  :rtype: NewTopic\n"

Answer 1

An example configuring max.message.bytes

import traceback

from confluent_kafka.admin import AdminClient, NewTopic
from confluent_kafka.cimpl import KafkaError

admin_client = AdminClient({"bootstrap.servers": "bootstrap.kafka:9092"})

topic_list = []

# topic: str,
# num_partitions: int,
# replication_factor: int = None,
# *args: Any,
# **kwargs: Any) -> None


replica_assignment = []
topic_config = {"max.message.bytes": 15048576}

topic_list.append(NewTopic(
    topic="example_topic",
    num_partitions=1,
    replication_factor=1,
    # replica_assignment=replica_assignment,
    config=topic_config
)
)

futures = admin_client.create_topics(topic_list)

try:
    record_metadata = []
    for k, future in futures.items():
        # f = i.get(timeout=10)
        print(f"type(k): {type(k)}")
        print(f"type(v): {type(future)}")
        print(future.result())

except KafkaError:
    # Decide what to do if produce request failed...
    print(traceback.format_exc())
    result = 'Fail'
finally:
    print("finally")