How to make a fixed-size byte variable in Python

Question

Let's say, I have a string (Unicode if it matters) variable which is less than 100 bytes. I want to create another variable with exactly 100 byte in size which includes this string and is padded with zero or whatever. How would I do it in Python 3?

Answer 1

For assembling packets to go over the network, or for assembling byte-perfect binary files, I suggest using the struct module.

• struct — Interpret bytes as packed binary data

Just for the string, you might not need struct , but as soon as you start also packing binary values, struct will make your life much easier.

Depending on your needs, you might be better off with an off-the-shelf network serialization library, such as Protocol Buffers; or you might even just use JSON for the wire format.

• Protocol Buffer Basics: Python
• PyMOTW - JavaScript Object Notation Serializer

Answer 2

Something like this should work:

st = "具有"
by = bytes(st, "utf-8")
by += b"0" * (100 - len(by))
print(by)
# b'\xe5\x85\xb7\xe6\x9c\x890000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000'

Obligatory addendum since your original post seems to conflate strings with the length of their encoded byte representation: Python unicode explanation

Answer 3

You could use the bytes.zfill method to add the required number of zeroes:

In [19]: result = bytes('おくりびと', 'utf-8').zfill(100)

In [20]: result
Out[20]: b'0000000000000000000000000000000000000000000000000000000000000000000000000000000000000\xe3\x81\x8a\xe3\x81\x8f\xe3\x82\x8a\xe3\x81\xb3\xe3\x81\xa8'

In [21]: len(result)
Out[21]: 100

Answer 4

To pad with null bytes you can do it the way they do it in the stdlib base64 module.

some_data = b'foosdsfkl\x05'
null_padded = some_data + bytes(100 - len(some_data))

Answer 5

Here's a roundabout way of doing it:

>>> import sys
>>> a = "a"
>>> sys.getsizeof(a)
22
>>> a = "aa"
>>> sys.getsizeof(a)
23
>>> a = "aaa"
>>> sys.getsizeof(a)
24

So following this, an ASCII string of 100 bytes will need to be 79 characters long

>>> a = "".join(["a" for i in range(79)])
>>> len(a)
79
>>> sys.getsizeof(a)
100

This approach above is a fairly simple way of "calibrating" strings to figure out their lengths. You could automate a script to pad a string out to the appropriate memory size to account for other encodings.

def padder(strng):
    TARGETSIZE = 100
    padChar = "0"

    curSize = sys.getsizeof(strng)

    if curSize <= TARGETSIZE:
        for i in range(TARGETSIZE - curSize):
            strng = padChar + strng

        return strng
    else:
        return strng  # Not sure if you need to handle strings that start longer than your target, but you can do that here