How can I initate retries on producer.poll() from the confluent_kafka library?

Question

I'm very new to using kafka, not sure if I understand it yet tbh. However, I'm tasked with making sure it will retry if there is an error of any kind. I've done this is with an API if I recieve a response other than 200 for example, but I'm stumpped with this kafka stuff. Here is my code below if it helps:

def send_kafka_message(payload, secrets):
"""Parses payload into message and sets on kafka producer."""
logger.info("Sending message on Kafka topic...")
producer = Producer(
    {
        "bootstrap.servers": CONFLUENT_SERVER,
        "sasl.mechanism": "PLAIN",
        "security.protocol": "SASL_SSL",
        "sasl.username": CONFLUENT_USER,
        "sasl.password": secrets["confluent_secret"],
        "error_cb": error_cb,
    }
)
producer.produce(
    ORDER_TOPIC,
    value=json.dumps(payload),
    callback=delivery_report,
    partition=abs(hash(payload["_id"])) % 6,
)
producer.poll()

Answer 1

The default retries amount is 2147483647

Description: How many times to retry sending a failing Message . Note : retrying may cause reordering unless enable.idempotence is set to true .

You can find all librdkafka settings, which are used for confluent-kafka-python here .