在python中将8位列表转换为32位整数数组

Question

what I have :

textdata = "this is my test data"
DataArray = [ord(c) for c in textdata]

now I want to transform this is into x 32 bit integer by combining 4 elements of the list together

Ex : DataArray[0:4] would become a 32 bit integer, and then iterate to the next 4 elements and do the same. In the end, I would have a 32-bit array with all my results in it.

How can I do this in python whitout iterating over the whole string. Is there a simple way to do this?

Answer 1

As long as your string is an integer multiple of 4, you can use NumPy in a very efficient way:

import numpy as np
data = np.fromstring(textdata, dtype='>u4')
# array([1952999795,  543781664, 1836654708, 1702065184, 1684108385])

'>u4' means 'big-endian unsigned 4-byte integer'.

Edit : If you use NumPy >= 1.14, then np.fromstring is deprecated, and the right way to process your text is by calling np.frombuffer(textdata.encode(), dtype='>u4') .

Answer 2

Using numpy:

>>> import numpy as np

>>> a = np.frombuffer(b'this is my test data', dtype=np.int32)
>>> a
array([1936287860,  544434464, 1948285293,  544502629, 1635017060], dtype=int32)
>>> a.tobytes()
b'this is my test data'

Use '<i4' or similar as dtype for endianness that's portable between machines.

I'm assuming that you can keep your initial data as bytes rather than unicode , because you really should try hard to do that.

Answer 3

You can use the struct built-in python module:

from struct import unpack

textdata = "this is my test data"
data = list(unpack('i'*(len(textdata)//4), textdata))

Result:

[1936287860, 544434464, 1948285293, 544502629, 1635017060]

You won't need to iterate over the string and you can find other Format Characters if you want to use unsigned integers for example.

Answer 4

You could use something like the following, which uses bit manipulation (big-endian):

def chunk2int(chunk):
    """ Converts a chunk (string) into an int, 8 bits per character """
    val = 0
    for c in chunk:
        val = (val << 8) | (ord(c) & 0xFF)
    return val

def int2chunk(val):
    """ Converts an int into a chunk, consuming 8 bits per character """
    rchunk = []
    while val:
        rchunk.append(val & 0xFF)
        val >>= 8

    return ''.join(chr(c) for c in reversed(rchunk))

textdata = "this is my test data"

chunks = [textdata[i:i + 4] for i in range(0, len(textdata), 4)]
print(chunks)

data = [chunk2int(c) for c in chunks]
print(data)

chunks = [int2chunk(d) for d in data]
print(chunks)

Produces:

['this', ' is ', 'my t', 'est ', 'data']
[1952999795, 543781664, 1836654708, 1702065184, 1684108385]
['this', ' is ', 'my t', 'est ', 'data']

If you're using characters with 1 <= ord(c) <= 255 in your input text, this will work. If there are null bytes in your string, the int2chunk method may terminate early, in which case you'd have to pad the chunks.

There's also the struct module, which may be worth looking into, and where you can change the endianness much more simply.