How do I read in hex values from a binary file and decipher some bytes containing bitflag values?

Question

I'm reading a bunch of bytes from a binary file. It is a RAR file. I'm interested in the 11th and 12th bytes of the file because the header specifications state:

HEAD_FLAGS Bit flags: 2 bytes

  0x0001 - Volume attribute (archive volume) 0x0002 - Archive comment present RAR 3.x uses the separate comment block and does not set this flag. 0x0004 - Archive lock attribute 0x0008 - Solid attribute (solid archive) 0x0010 - New volume naming scheme ('volname.partN.rar') 0x0020 - Authenticity information present RAR 3.x does not set this flag. 0x0040 - Recovery record present 0x0080 - Block headers are encrypted 0x0100 - First volume (set only by RAR 3.0 and later) other bits in HEAD_FLAGS are reserved for internal use 

The file that I'm playing with has 00 and 0D as positions 11 and 12 respectively.

I can't make sense of these two values as they are bit flags (which I have failed to understand).

I have these two values in byte array that is 12 bytes long. What I need to check in this sequence is whether the flag 0x0100 and 0x0001 is set or not.

I'm lost with this. Thanks.

---

I've inspected some of the files in a Hex editor and what i've seen is that 11th and 12th bytes need to be read together. That's why the specs list all the bit flags are 4 letter hex codes. Checking the bit flags individually yields incorrect results.

---

Assimilating as much information from the answers/tips, 've solved this in the following way:

FileInputStream fisFileInputStream = new FileInputStream((new File("C:\\testarchive.r00"));

byte[] bytHeader = new byte[20]; //The first 20 bytes are the RAR header.
fisFileInputStream.read(bytHeader);

short val=(short)( ((bytHeader[10]&0xFF)<<8) | (bytHeader[11]&0xFF) ); //Joining the two bytes into a short

System.out.println("Volume Attribute (0x0001): " + ((val & 0x0001) != 0));
System.out.println("First volume (0x0100): " + ((val & 0x0100) != 0));

I've tried this code with multiple RAR archives — spanned ones, non-spanned ones, the first file of a spanned archive, another file of a spanned archive.

The code itself works fine except for a very minor quirk. I get the opposite results from my hex values ie

When inspecting a file which is not the first file in a spanned archive, I get the volume atrribute 0x0001 as not set ( false ), and the "First volume" 0x100 as set ( true ).

When inspecting a file which is the first file in a spanned archive, I get the exact opposite results.

Now I modify my code to believe that the original specs are wrong (highly unlikely) and that 0x0001 means that is is the first file in a spanned archive and 0x0100 means that is a spanned archive, then it's all okay.

..but I guess I'm doing something wrong here with my bit flag logic. Any ideas?

Answer 1

The idea of bit flags is that you can cram a bunch of flags into a small number of bytes by adding them together.

In this case, they're not saying that each value you see in that list is a separate pair of bytes. They're all crammed into two bytes. If you were writing the file, you would add these values together to get the value to write.

For example, suppose we want to say "Archive lock attribute", "New volume naming scheme", and "Block headers encrypted". That means we want to set 0x0004, 0x0010, and 0x0080. So we add these all together, 0x0004 + 0x0010 + 0x0080 = 0x0094, and that's the value we write. For this to work, all values must turn out to be a single bit, or to put it another way, all must be powers of 2.

So this documentation doesn't describe 18 bytes. It only describes 2.

To read it, you have to perform AND operations (&) with the desired flag value. Life if you want to know if "Block headers encrypted" is set, read the value and AND with 0x0010. As this is a two-byte value, you either want to get it into a short or you have to pick the correct byte. Let's say for the sake of argument that we get it into a short. Then we'd say

if ((flags & 0x0010) != 0)

If that expression is true, then the bit is set. If it is false, the bit is not set.

BTW, if you read it as a stream of bytes, you can put them into a short by writing:

short flags = b[0] | b[1]<<8;

(You may need a cast in there. I forget.)

Or you may have to switch b[0] and b[1], depending on whether the file is written low-hi or hi-low.

Answer 2

      FileInputStream fis = new FileInputStream("file.rar");
      byte[] ba = new byte[13];
      fis.read(ba);
      byte b11 = ba[11];
      byte b12 = ba[12];
      boolean flagIs0x10 = b12 == 0x10;
      System.out.println("flag is 0x10 = "+flagIs0x10);

If both bytes together should have the value 0x10, ie if it is a 16bit-word, then

      boolean flagIs0x10 = b11 == 0 && b12 == 0x10;

or

      boolean flagIs0x10 = b12 == 0 && b11 == 0x10;

Answer 3

Here's an easy way to deal with the arithmetic. The following utility function modifies a java BitSet to set bit flags based on the contents of a byte value.

static void setByte(BitSet bitSet, int byteNum, byte b) {
    int base = byteNum * 8;
    bitSet.set(base + 7, (b & 0x80) != 0);
    bitSet.set(base + 6, (b & 0x40) != 0);
    bitSet.set(base + 5, (b & 0x20) != 0);
    bitSet.set(base + 4, (b & 0x10) != 0);
    bitSet.set(base + 3, (b & 0x08) != 0);
    bitSet.set(base + 2, (b & 0x04) != 0);
    bitSet.set(base + 1, (b & 0x02) != 0);
    bitSet.set(base + 0, (b & 0x01) != 0);
}

It takes a "byteNum" parameter, in case you want to have multiple bytes put into the bitset, which in your case you do.

byte 0 would be bitset positions 0-7 byte 1 would be bitset positions 8-15 etc....

And within each byte, the high order bit is position 7, low order bit position 0.

Then you could simply do this once you have the two bytes. I'm assuming that the first byte is the high order bits, (0x0800 - 0x8000), and the second byte is the low order bits (0x0001 - 0x0080), though your spec should tell you that.

byte buf[] = new byte[2];
inputStream.read(buf, 0, 2);

BitSet bitSet = new BitSet();
// low order byte is the second one
setByte(bitSet, 0, bytes[1]);
// high order byte is first
setByte(bitSet, 1, byte1[0]);

boolean archiveVolume = bitSet.get(0);
boolean commentPresent = bitSet.get(1);
...
boolean firstVolume = bitSet.get(8);