Extracting bits from bytes

Question

I am handling compressed data. The format contains a lookup table, and an array of long ints, which each may contain multiple values. Bit length of contained values varies depending on the file. I have access to these longs as bytes : is there an easy way to access a particular bit / bit range, or do I have to make one from scratch ? Within the standard library

Note that I may have to continue to the next long value when the bit length isn't a factor of 64.

---

Theoretical example of what the code needs to do :

• Take the long integer 4503672641818897L
• Convert it to bits ( should return 0000000000010000000000000001000100000000000000000001000100010001 )
• Read the lookup table and determine how long the values are (let's say 5 bits this time)
• Read the 6th value, bits 25 - 29 ( 00100 )
• Return the int value 4

Answer 1

Here is a solution not depending on external libraries like numpy or on string conversions:

def get_bits(num, start, end, length=64):
    '''Like bits(num)[from:to] interpreted as int'''
    mask = 2**(end-start)-1
    shift = length - (end-start) - start
    return (num & (mask << shift)) >> shift


print(get_bits(17, 0, 3, length=6))  # 010001[0:3] -> 010 = 2
print(get_bits(17, 3, 6, length=6))  # 010001[3:6] -> 001 = 1
print(get_bits(17, 0, 6, length=6))  # 010001[0:6] -> 010001 = 17
print(get_bits(4503672641818897, 25, 30))  # ...[25:30] -> 00100 = 4

Explanation:

• mask = 2**(end-start)-1 : end-start is the number of bits to select (N), then 2**N is a one with N zeros (2**3 -> 1000). 2**N - 1 then is N ones (1000 - 1 = 111).
• shift = length - (end-start) - start : The number of bits we want to shift the mask to the left (111 << 3 = 111000) and also the number of bits we want the result to shift to the right: 010001 & 111000 is 010000, we only want the first three bits. 010000 >> 3 is 010.
• return (num & (mask << shift)) >> shift : Now we put it all together

Answer 2

试试这个功能unpackbits从numpy https://numpy.org/doc/stable/reference/generated/numpy.unpackbits.html

Bits = numpy.unpackbits(Bytes)

Answer 3

What about this solution:

unpacked = "{0:b}".format(long_int)
unpacked = "0"*(64-len(unpacked)) + unpacked
int(unpacked[25:30],2)

Answer 4

EDIT : DOES NOT WORK ! the int constructor assumes a signed int, and there is no way to tell it to construct a uint

Here's a hacky solution I found. Seems very unwieldy though

def bitValue(byteValue, start, length):
    """Extract length bits from byteValue at start, and return them as an integer"""
    return int(bin(byteValue).lstrip('0b').rjust(64, '0')[start:start+length], 2)