python 3 mac：snappy.compress AttributeError：模块“snappy”没有属性“compress”

Question

Has anyone solved the error: message: compressions['SNAPPY'] = snappy.compress AttributeError: module 'snappy' has no attribute 'compress' when reading parquet in python? Btw, is there a way to read whole dir?

I am using python 3 through conda on mac with snappy and thrift installed as per https://pypi.python.org/pypi/parquet

code as follows:

import parquet
import json
import fastparquet

with open(data_in_path + "file.parquet/part-01snappy.parquet", 'rb') as fo:
for row in parquet.DictReader(fo, columns=['id', 'title']):
    print(json.dumps(row))

or

 df2 = fastparquet.ParquetFile(path).to_pandas()

Answer 1

I had the same issue.

The reason was I installed the wrong python package. You should install python-snappy instead of snappy

In my case (os x), it was a simple, two step process:

brew install snappy
pip install python-snappy

Answer 2

无法找到snappy解决方案，所以我用snappy在spark读取数据，并在发现 python 中的每个问题后用gzip将其写回：

df.coalesce(1).write.option("overwrite","true").option("compression","gzip").parquet(dfWithGzip.parquet")

Answer 3

I was experiencing the same issue. uninstalling and reinstalling as @Javier Alba specified, it worked for me.