将base64编码的字符串存储在HBase中

Question

I have a very specific requirement of storing PDF data in Hbase columns. The source of Data is Mongo DB, from where the base64 encoded data is read and I will need to bulk upload it to Hbase table.

I realized that in base64 encoded string there are a lot of "\\n" character which splits the entire string into parts. Not sure if it is because of this, but when I store the string as it is, using a put :

 put.add(Bytes.toBytes(ColFamilyName), Bytes.toBytes(columnName), Bytes.toBytes(data.replaceAll("\n","").toString()));

It is storing only the first line from the entire encoded string. Eg :

If the actual content was something like this : "JVBERi0xLjQKJaqrrK0KNCAwIG9iago8PAovQ3JlYXRvciAoQXBhY2hlIEZPUCBWZXJzaW9uIDEu " + "MSkKL1Byb2R1Y2VyIChBcGFjaGUgRk9QIFZlcnNpb24gMS4xKQovQ3JlYXRpb25EYXRlIChEOjIw\\n" + "MTUwODIyMTIxMjM1KzAzJzAwJykKPj4KZW5kb2JqCjUgMCBvYmoKPDwKICAvTiAzCiAgL0xlbmd0\\n" +

It is storing only the first line which is : JVBERi0xLjQKJaqrrK0KNCAwIG9iago8PAovQ3JlYXRvciAoQXBhY2hlIEZPUCBWZXJzaW9uIDEu

in the column. Even after trying to remove the "\\n" manually it is the same output.

Could someone please guide me in the right direction here ?

Answer 1

Currently, I am also working on Base64 encoding. As per my understanding, you should try using org.apache.hadoop.hbase.util.Base64.encodeBytes(byte[] source, int option) method where DONT_BREAK_LINES can be used as an option. Please let me know if this works fine.

Answer 2

Managed to solve it. The issue was when reading the Base64 encoded data from MongoDB Source. Read the data from Mongo DB document DBObject as:

jsonObj.get("receiptContent").toString().replaceAll("\\n","")

And stored it as such in Hbase. Even from the Hue HBase UI Browser I can see the PDF content now.