Parsing bits from a 128 byte block of hex in Python

Question

Ok, so I am in a bit of a weird parsing scenario, but here it goes.

I have a script that reads in the bytes it needs to parse. I need to parse out those bytes and then return them.

Example

-------------------------------------------------------------------
Description: Log Parameters   : Byte Offset:  0
-------------------------------------------------------------------
-------------------------------------------------------------------
Description: Offset           : Byte Offset:  2-1
-------------------------------------------------------------------
-------------------------------------------------------------------
Description: Request Count    : Byte Offset:  3
-------------------------------------------------------------------
-------------------------------------------------------------------
Description: Reserved         : Byte Offset:  127-4
-------------------------------------------------------------------

So my script will eventually have the ability to output the hex associated with each line. For now, I need to say, ok, Byte offset is 0, go get the first byte and return it in hex. Ok, byte offset is 127-4, go get that, print the hex value right there on the screen.

The format is 127 bytes of hex stored in a string.

HEX String

100000000000000220000000000000003000000000000000
000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000
0000000000000000

The 0x prefix has been stripped and stored into a string.

There are a lot of zeroes in this example, but this is just a random case. The byte offsets can fluctuate, so I'm trying to figure out how to basically iterate through an array of byte offsets and parse them incrementally.

It's weird to me that if a description takes up so many bytes, bitwise operations become more difficult because I can't split these up into 32 or even 64 bit blocks.

What I Want

Currently I have an array of the byte offsets in the following form:

[0, 2-1, 3, 127-4]

I want to iterate through each of those byte offsets, parse them from long hex string and print them.

Question

How do I use the byte offsets from my array and parse them out of the hex string.

Answer 1

Say that you have the starting byte # stored in start variable, and ending byte # stored in end variable, and then the hex string stored in string variable.

Since every byte is two hexadecimal digits, you can simply do this to get the byte in hexadecimal string form:

string[start*2:(end+1)*2]

You need to do end+1 because it appears that your byte ranges are inclusive in your example, but Python slicing is exclusive on the end of the range. More on slicing if you're unfamiliar.

To make this concrete for you, here is a minimal working example. You may have to do parsing and massaging to get your ranges to look like mine, but this is the idea:

string = "100000000000000220000000000000003000000000000000" \
         "000000000000000000000000000000000000000000000000" \
         "000000000000000000000000000000000000000000000000" \
         "000000000000000000000000000000000000000000000000" \
         "000000000000000000000000000000000000000000000000" \
         "0000000000000000"

ranges = ['0', '2-1', '3', '127-4']

for offset in ranges:
    offset_list = offset.split('-')
    if len(offset_list) == 1:
        start = int(offset_list[0])
        end = int(offset_list[0])
    else:
        start = int(offset_list[1])
        end = int(offset_list[0])
    the_bytes = string[start*2:(end+1)*2]
    print('%d-%d: %s' % (start, end, the_bytes))

Output:

0-0: 10
1-2: 0000
3-3: 00
4-127: 00000002200000000000000030000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Answer 2

# Input: array of byte values
x='''
100000000000000220000000000000003000000000000000
000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000
0000000000000000
'''

# Input: list of offsets
o = ['0', '2-1', '3', '127-4']

# Put everything in a more useful format
x = ''.join(x.split())
o = [item.split('-') for item in o]
o = [[int(item) for item in pair] for pair in o]
for pair in o:
    if len(pair) == 1:
        pair.append(pair[0])

# Display the values
for pair in o:
    print pair, x[pair[1]*2:pair[0]*2+2]