简体繁体 English

Hadoop GPG SerDe

[英]Hadoop GPG SerDe

原文 2012-08-29 16:28:30 9 1 encryption/ hadoop/ hive/ gnupg

I am currently working on a Hadoop project that requires data encryption (because the data will be stored in S3). 我目前正在从事需要数据加密的Hadoop项目（因为数据将存储在S3中）。 While I primarily expect to access the data though Hive, it would be nice to be able to access it via Pig and any other MapReduce methods. 虽然我主要希望通过Hive访问数据，但能够通过Pig和任何其他MapReduce方法访问数据将是一个不错的选择。

I know Hadoop has built-in support for compression codecs like gzip, snappy, etc... Is there any support for encryption codecs as well (specifically, GPG)? 我知道Hadoop内置了对压缩编解码器（如gzip，snappy等）的支持。是否也对加密编解码器（特别是GPG）提供支持？ Has anyone written a GPG SerDe (or anything similar) that is publicly available? 是否有人编写了公开可用的GPG SerDe（或类似内容）？

1 个解决方案

Last I knew Hadoop has no internal support for encryption whatsoever. 最后，我知道Hadoop没有任何内部加密支持。 Seems like you could overload the CompressionCodec with your GPG code, ala http://www.mail-archive.com/common-user@hadoop.apache.org/msg06229.html 似乎您可以使用GPG代码重载CompressionCodec，例如http://www.mail-archive.com/common-user@hadoop.apache.org/msg06229.html

Happy Hacking & let us know if you find a solution! Happy Hacking，如果您找到解决方案，请告诉我们！