Python - Better way to convert string and escaped hex string data to integers?

Question

I'm working on a project where I receive data from an XBEE radio. The data comes in different strings... I've finally found a way to extract the actual information however, I really think there has got to be a better way to convert the data into usable form.

Here is an example of a frame:

 {'analog_ch_mask': '\x18', 'first_sample': '\x03Q', 'number_samples': '\x01', 
 'options': '\xc1', 'addr_short': '\xff\xfe', 
 'address_low': '@c{!', 'digital_ch_mask': '\x00\x00', 
 'address_high':  '\x00\x13\xa2\x00', 'second_sample': '\x00\xca', 'id': 'io_sample_rx'}

The issue I've encountered is the formatting of the data, the following works for me.

# Extract the characters and join them together.
sample_data = ''.join("{:02X}".format(ord(c)) for c in item['key'])
print(sample_data)
print type(sample_data) # is a string
# Convert the hex string into an integer
sample_data = int(sample_data, 16)
print(sample_data)
print type(sample_data) # is an integer
# Try converting to hex string just for fun
sample_data = hex(sample_data)
print(sample_data)
print type(sample_data) # is a string

I like that this works for both the ascii data as well as the escape hex strings. However, shouldn't there be a more direct way to do these operations? I attempted using unpack, but I was getting errors.

Cheers.

Answer 1

Try to use struct module for unpacking:

import struct


frame = {'analog_ch_mask': '\x18', 'first_sample': '\x03Q', 'number_samples': '\x01', 
'options': '\xc1', 'addr_short': '\xff\xfe', 
'address_low': '@c{!', 'digital_ch_mask': '\x00\x00', 
'address_high':  '\x00\x13\xa2\x00', 'second_sample': '\x00\xca', 'id': 'io_sample_rx'}

print 'analog_ch_mask: %s' % struct.unpack('>B', frame['analog_ch_mask'])[0]
print 'first_sample: %s' % struct.unpack('>H', frame['first_sample'])[0]
print 'number_samples: %s' % struct.unpack('>B', frame['number_samples'])[0]
print 'options: %s' % struct.unpack('>B', frame['options'])[0]
print 'addr_short: %s' % struct.unpack('>H', frame['addr_short'])[0]
print 'address_low: %s' % struct.unpack('>I', frame['address_low'])[0]
print 'digital_ch_mask: %s' % struct.unpack('>H', frame['digital_ch_mask'])[0]
print 'address_high: %s' % struct.unpack('>I', frame['address_high'])[0]
print 'second_sample: %s' % struct.unpack('>H', frame['second_sample'])[0]
print 'id: %s' % frame['id']

Here >B , >H and >I formats with big-endian byte order for unsigned char (1 byte) , unsigned short (2 bytes) , unsigned int (4 bytes) respectively are used.

The output is:

analog_ch_mask: 24
first_sample: 849
number_samples: 1
options: 193
addr_short: 65534
address_low: 1080261409
digital_ch_mask: 0
address_high: 1286656
second_sample: 202
id: io_sample_rx

PS It is possible that other endianness needed.