Encoding and decoding random byte array with zxing

Question

I'm trying to transfer a byte array with QR code, so for testing, I decided to generate a random byte array, encode it as QR code, then decode it. I used ISO-8859-1 to convert byte array to string st it does not lose data while transmission:

For encoder side:

byte []buffer = new byte[11];
com.google.zxing.Writer writer = new QRCodeWriter();
Random randomGenerator = new Random();
    for(int i=0;i<=10;i++){
        buffer[i]=(byte) randomGenerator.nextInt(254);
    }
   // Log.i("time1","original: "+Arrays.toString(buffer));
    String decoded = null;
    try {
        decoded = new String(buffer, "ISO-8859-1");
    } catch (UnsupportedEncodingException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
    try {
        result=writer.encode(decoded, BarcodeFormat.QR_CODE, 500, 500);
    } catch (WriterException e1) {
        // TODO Auto-generated catch block
        e1.printStackTrace();
    }

In this way I have converted byte array to QR code, it has no problem. But for the receiver side:

LuminanceSource source = new PlanarYUVLuminanceSource(data,640,480,0,0,640,480,false);
bmtobedecoded = new BinaryBitmap(new HybridBinarizer(source));
Map<DecodeHintType,Object> mp=new HashMap<DecodeHintType, Object>();

mp.put(DecodeHintType.TRY_HARDER, true);
try {
    result= qrr.decode(bmtobedecoded,mp);
} catch (NotFoundException e) {
    Log.i("123","not found");
    e.printStackTrace();
} catch (ChecksumException e) {
    Log.i("123","checksum");
    e.printStackTrace();
} catch (FormatException e) {
    Log.i("123","format");
    e.printStackTrace();
}

I tried to decode the generated QR code, but it throws out NotFoundException. Can someone help me with this issue?

Update 1 : I confirmed that the decoder works perfectly with the normal QR, I also added DecodeHintType.try_harder but still no good.

Update 2 : To clarify, below is what I did to convert between byte array and string:

Random randomGenerator = new Random();
for(int i=0;i<=10;i++){
    buffer[i]=(byte) randomGenerator.nextInt(254);
}
Log.i("time1","original: "+Arrays.toString(buffer));
String decoded = null;
try {
    decoded = new String(buffer, "ISO-8859-1");
} catch (UnsupportedEncodingException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
}
Log.i("time1","encoded string:" + decoded);
BitMatrix result=null;
try {
    result=qw.encode(decoded, BarcodeFormat.QR_CODE, 500, 500);
} catch (WriterException e1) {
    // TODO Auto-generated catch block
    e1.printStackTrace();
}
iv.setImageBitmap(encodematrix(result));
byte[] encoded = null;
try {
    encoded = decoded.getBytes("ISO-8859-1");
} catch (UnsupportedEncodingException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
} 
Log.i("time1","result byte array:" + java.util.Arrays.toString(encoded));

If you run this, you can easily see that you can get exactly the same array in the end. I have no problem with this.

Update 3 : I also tried encoding it using UTF-8, but it loses data, so it cannot be the used in encoder.

Update 4 : Just added:

Map<DecodeHintType,Object> mp=new HashMap<DecodeHintType, Object>();
mp.put(DecodeHintType.CHARACTER_SET, "ISO-8859-1");

in the decoder, still throwing out exception.

Answer 1

Try PURE_BARCODE mode as a detection hint. Strangely, false positive detection of finder patterns is a much bigger problem when the image is just a pure synthetic image. The heuristics assume a photo, which doesn't have these problems. In this alternate mode it can take advantage of knowing it's a pure image and not a photo and be much faster and never get the detection wrong.

Answer 2

There are two issues that you have to overcome to store binary data in QR codes.

1. ISO-8859-1 does not allow bytes in ranges of 00-1F and 7F-9F. Since you are using a random generator, you can just check whether a randomly generated byte fits this value and re-generate it until you get a random byte that fits this range. If you nevertheless need to encode these bytes anyway, you may encode the array as a Base-64 string or as a hexadecimal string. In case of a hexadecimal string, it will be stored in the QR code in the alphanumeric mode, not in 8-bit mode.

2. Since you are trying to store binary data in QR codes, you have to rely only on your own scanner that will handle this binary data, and you need to make sure that your scanner does not use heuristics to automatically determine character encoding, and so on. Most QR decoders use heuristics to detect the character set used. These heuristics may detect a character set other than ISO-8859-1 and thus fail to properly display your binary data. Some scanners use heuristics to detect a character set even if the character set is explicitly given by the ECI optional extension inside the QR Code.

So, using US-ASCII only (eg, binary data encoded in Base64 before passing it to a QR Code generator) is the safest choice for QR code against the heuristics. This will also overcome another complication: that ISO-8859-1 was not the default encoding in earlier QR code standard published in 2000 (ISO/IEC 18004:2000). That standard did specify 8-bit Latin/Kana character set in accordance with JIS X 0201 (JIS8 also known as ISO-2022-JP) as default encoding for 8-bit mode, while the updated standard published in 2005 did change the default to ISO-8859-1.

If you store your buffer as a hexadecimal string in your QR code, this will disable all heuristics for sure and should not produce larger QR Code than with Base-64, because each character in the alphanumeric mode takes only 6 bits in the QR code stream.