LongSerializer vs ByteArraySerializer in kafka

Question

I am a new to Kafka and trying to understand best way to use few things. I have a topic which will have 10 partitions and I am sending data to it using kafka producer. My key will be client_id which is Long data type and value will be byte array. So should I use LongSerializer for my key serializer (key.serializer) or I should convert my client_id to byte array in my code itself and then use ByteArraySerializer for key.serializer instead?

So question is what is the difference between directly using LongSerializer vs using ByteArraySerializer . In both the case, it converts long to byte array?

I am running Kafka 0.10.0.0 version.

Answer 1

LongSerializer's serialize method only accepts a Long/long type parameter, which converts the long-typed integer into a byte array internally. If you use ByteArraySerializer, you have to do this conversion before passing around to ByteArraySerializer.

Answer 2

The different is that the send() on the producer takes a ProducerRecord with a Long vs a byte array. You're correct in that at some level it's all the same thing, but usually you like the ProducerRecord types to match what your user's code is using without additional conversions.

Answer 3

My key will be client_id which is Long data type and value will be byte array

It is clear, that for key you should be using LongSerializer and for value ByteArraySerializer

LongSerializer vs using ByteArraySerializer

As said, in the above answers, any object can (and will) be serialized to a byte[] , including your Long object. So, even the LongSerializer internally does that only.

public class LongSerializer implements Serializer<Long> {

    public void configure(Map<String, ?> configs, boolean isKey) {
        // nothing to do
    }

    public byte[] serialize(String topic, Long data) {
        if (data == null)
            return null;

        return new byte[] {
            (byte) (data >>> 56),
            (byte) (data >>> 48),
            (byte) (data >>> 40),
            (byte) (data >>> 32),
            (byte) (data >>> 24),
            (byte) (data >>> 16),
            (byte) (data >>> 8),
            data.byteValue()
        };
    }

    public void close() {
        // nothing to do
    }
}

So, better go with LongSerializer for you key (so that you need not worry about conversion). Why re-invent the wheel?