Java: correct way to cast “unsigned” byte to int

Question

I have spent several hours on looking for bug in third-party implementation of stream cipher Rabbit . There are several bugs, but one of them:

/**
 * @param IV An array of 8 bytes
 */
public void setupIV(final byte[] IV) {
    short[] sIV = new short[IV.length>>1];
    for(int i=0;i<sIV.length;++i) {
        sIV[i] = (short)((IV[i << 1] << 8) | IV[(2 << 1) + 1]);
    }
    setupIV(sIV);
}

The problem here is that byte IV[i << 1] is casted to int , but since Java doesn't have unsigned types, any value >= 0x80 is casted wrong. Say, byte 0xff becomes 0xffffffff , not 0x000000ff how author of the code above expected.

So I have added simple function:

   private int byte2int(byte b){
      int ret = b;
      if (ret < 0){
         ret += 256;
      }
      return ret;
   }

It works, but I'm wondering if this is correct way to do what I need? This solution seems dumb somewhat.

Answer 1

I'm not sure how much this will help, but you may convert a signed byte to its unsigned value by running a binary AND operation with 0xFF .

Taking this logic further, you could check for overflows after the cast by running a similar AND on the int, with the appropriate operand. This of course assumes that you're always expecting positive or zero valued numbers, or in other words unsigned numbers.

(short)(((IV[i << 1] & 0xFF) << 8) | (IV[(2 << 1) + 1] & 0xFF))

The above is the result of putting it all together, courtesy of Radiodef .

Answer 2

Use library functions for this operation (in ByteBuffer class). You will be able to control endianness as a bonus. This will rely on efficiency of Java creators.

package tests.StackOverflow.q20776371;

import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.ShortBuffer;

public class q20776371 {


    public static void main(String[] args) {

        byte[] bb = { (byte)0xFF, (byte)0x01, (byte)0x02, (byte)0x03 };

        ByteBuffer buffer1 = ByteBuffer.wrap(bb);

        // set endianness
        // buffer1.order(ByteOrder.BIG_ENDIAN);
        // buffer1.order(ByteOrder.LITTLE_ENDIAN);

        ShortBuffer buffer2 = buffer1.asShortBuffer();

        short[] ss = new short[bb.length>>1];
        buffer2.get(ss);

        for(int i=0; i<ss.length; ++i) {
            System.out.println(String.format("%04x", ss[i]));
        }
    }


}