kafka.errors.UnsupportedCodecError: UnsupportedCodecError: kafka consumer 时找不到 snappy 压缩编解码器的库

Question

Getting below error in message in consumer line

code:

consumer = KafkaConsumer(topic,group_id=groupid,bootstrap_servers=[host]) for message in consumer:

Error: for message in consumer: File " \Python\Python38-32\lib\site-packages\kafka\consumer\group.py", line 1192, in next return self.next_v2() File " \Python\Python38-32\lib\site-packages\kafka\consumer\group.py", line 1200, in next_v2 return next(self._iterator) File " \Python\Python38-32\lib\site-packages\kafka\consumer\group.py", line 1115, in _message_generator_v2 record_map = self.poll(timeout_ms=timeout_ms, update_offsets=False) File " \Python\Python38-32\lib\site-packages\kafka\consumer\group.py", line 654, in poll records = self._poll_once(remaining, max_records, update_offsets=update_offsets) File " \Python\Python38-32\lib\site-packages\kafka\consumer\group.py", line 707, in _poll_once records, _ = self._fetcher.fetched_records(max_records, update_offsets=update_offsets) File " \Python\Python38-32\lib\site-packages\kafka\consumer\fetcher.py", line 344, in fetched_records self._next_partition_records = self._parse_fetched_data(completion) File " \Python\Python38-32\lib\site-pa ckages\kafka\consumer\fetcher.py", line 816, in _parse_fetched_data unpacked = list(self._unpack_message_set(tp, records)) File " \Python\Python38-32\lib\site-packages\kafka\consumer\fetcher.py", line 467, in _unpack_message_set for record in batch: File " \Python\Python38-32\lib\site-packages\kafka\record\legacy_records.py", line 245, in iter self._buffer = memoryview(self._decompress(key_offset)) File " \Python\Python38-32\lib\site-packages\kafka\record\legacy_records.py", line 186, in _decompress self._assert_has_codec(compression_type) File "*Python\Python38-32\lib\site-packages\kafka\record\legacy_records.py", line 126, in _assert_has_codec raise UnsupportedCodecError( kafka.errors.UnsupportedCodecError: UnsupportedCodecError: Libraries for snappy compression codec not found

Answer 1

UnsupportedCodecError: Libraries for snappy compression codec not found

To consume snappy-compressed messages, you need to install snappy .

Alternatively, if you control the producer, don't use snappy.