kafka.errors.UnsupportedCodecError: UnsupportedCodecError: Libraries for snappy compression codec not found while kafka consumer
Question
Getting below error in message in consumer line
code:
consumer = KafkaConsumer(topic,group_id=groupid,bootstrap_servers=[host]) for message in consumer:
Error: for message in consumer: File " \Python\Python38-32\lib\site-packages\kafka\consumer\group.py", line 1192, in next return self.next_v2() File " \Python\Python38-32\lib\site-packages\kafka\consumer\group.py", line 1200, in next_v2 return next(self._iterator) File " \Python\Python38-32\lib\site-packages\kafka\consumer\group.py", line 1115, in _message_generator_v2 record_map = self.poll(timeout_ms=timeout_ms, update_offsets=False) File " \Python\Python38-32\lib\site-packages\kafka\consumer\group.py", line 654, in poll records = self._poll_once(remaining, max_records, update_offsets=update_offsets) File " \Python\Python38-32\lib\site-packages\kafka\consumer\group.py", line 707, in _poll_once records, _ = self._fetcher.fetched_records(max_records, update_offsets=update_offsets) File " \Python\Python38-32\lib\site-packages\kafka\consumer\fetcher.py", line 344, in fetched_records self._next_partition_records = self._parse_fetched_data(completion) File " \Python\Python38-32\lib\site-pa ckages\kafka\consumer\fetcher.py", line 816, in _parse_fetched_data unpacked = list(self._unpack_message_set(tp, records)) File " \Python\Python38-32\lib\site-packages\kafka\consumer\fetcher.py", line 467, in _unpack_message_set for record in batch: File " \Python\Python38-32\lib\site-packages\kafka\record\legacy_records.py", line 245, in iter self._buffer = memoryview(self._decompress(key_offset)) File " \Python\Python38-32\lib\site-packages\kafka\record\legacy_records.py", line 186, in _decompress self._assert_has_codec(compression_type) File "*Python\Python38-32\lib\site-packages\kafka\record\legacy_records.py", line 126, in _assert_has_codec raise UnsupportedCodecError( kafka.errors.UnsupportedCodecError: UnsupportedCodecError: Libraries for snappy compression codec not found
1 answers
solution1
3 2020-05-15 21:13:08
UnsupportedCodecError: Libraries for snappy compression codec not found
To consume snappy-compressed messages, you need to install snappy .
Alternatively, if you control the producer, don't use snappy.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.