Python：将字母数字字符串可逆地编码为整数

Question

I want to convert a string (composed of alphanumeric characters) into an integer and then convert this integer back into a string:

string --> int --> string

In other words, I want to represent an alphanumeric string by an integer.

I found a working solution, which I included in the answer, but I do not think it is the best solution, and I am interested in other ideas/methods.

Please don't tag this as duplicate just because a lot of similar questions already exist, I specifically want an easy way of transforming a string into an integer and vice versa .

This should work for strings that contain alphanumeric characters, ie strings containing numbers and letters.

Answer 1

Here's what I have so far:

string --> bytes

mBytes = m.encode("utf-8")

bytes --> int

mInt = int.from_bytes(mBytes, byteorder="big")

int --> bytes

mBytes = mInt.to_bytes(((mInt.bit_length() + 7) // 8), byteorder="big")

bytes --> string

m = mBytes.decode("utf-8")

try it out:

m = "test123"
mBytes = m.encode("utf-8")
mInt = int.from_bytes(mBytes, byteorder="big")
mBytes2 = mInt.to_bytes(((mInt.bit_length() + 7) // 8), byteorder="big")
m2 = mBytes2.decode("utf-8")
print(m == m2)

---

Here is an identical reusable version of the above:

class BytesIntEncoder:

    @staticmethod
    def encode(b: bytes) -> int:
        return int.from_bytes(b, byteorder='big')

    @staticmethod
    def decode(i: int) -> bytes:
        return i.to_bytes(((i.bit_length() + 7) // 8), byteorder='big')

If you're using Python <3.6, remove the optional type annotations.

Test:

>>> s = 'Test123'
>>> b = s.encode()
>>> b
b'Test123'

>>> BytesIntEncoder.encode(b)
23755444588720691
>>> BytesIntEncoder.decode(_)
b'Test123'
>>> _.decode()
'Test123'

Answer 2

Recall that a string can be encoded to bytes, which can then be encoded to an integer. The encodings can then be reversed to get the bytes followed by the original string.

This encoder uses binascii to produce an identical integer encoding to the one in the answer by charel-f. I believe it to be identical because I extensively tested it.

Credit: this answer .

from binascii import hexlify, unhexlify

class BytesIntEncoder:

    @staticmethod
    def encode(b: bytes) -> int:
        return int(hexlify(b), 16) if b != b'' else 0

    @staticmethod
    def decode(i: int) -> int:
        return unhexlify('%x' % i) if i != 0 else b''

If you're using Python <3.6, remove the optional type annotations.

Quick test:

>>> s = 'Test123'
>>> b = s.encode()
>>> b
b'Test123'

>>> BytesIntEncoder.encode(b)
23755444588720691
>>> BytesIntEncoder.decode(_)
b'Test123'
>>> _.decode()
'Test123'

Answer 3

Assuming the character set is merely alphanumeric, ie az AZ 0-9, this requires 6 bits per character. As such, using an 8-bit byte-encoding is theoretically an inefficient use of memory.

This answer converts the input bytes into a sequence of 6-bit integers. It encodes these small integers into one large integer using bitwise operations. Whether this actually translates into real-world storage efficiency is measured by sys.getsizeof , and is more likely for larger strings.

This implementation customizes the encoding for the choice of character set. If for example you were working with just string.ascii_lowercase (5 bits) rather than string.ascii_uppercase + string.digits (6 bits), the encoding would be correspondingly efficient.

Unit tests are also included.

import string


class BytesIntEncoder:

    def __init__(self, chars: bytes = (string.ascii_letters + string.digits).encode()):
        num_chars = len(chars)
        translation = ''.join(chr(i) for i in range(1, num_chars + 1)).encode()
        self._translation_table = bytes.maketrans(chars, translation)
        self._reverse_translation_table = bytes.maketrans(translation, chars)
        self._num_bits_per_char = (num_chars + 1).bit_length()

    def encode(self, chars: bytes) -> int:
        num_bits_per_char = self._num_bits_per_char
        output, bit_idx = 0, 0
        for chr_idx in chars.translate(self._translation_table):
            output |= (chr_idx << bit_idx)
            bit_idx += num_bits_per_char
        return output

    def decode(self, i: int) -> bytes:
        maxint = (2 ** self._num_bits_per_char) - 1
        output = bytes(((i >> offset) & maxint) for offset in range(0, i.bit_length(), self._num_bits_per_char))
        return output.translate(self._reverse_translation_table)


# Test
import itertools
import random
import unittest


class TestBytesIntEncoder(unittest.TestCase):

    chars = string.ascii_letters + string.digits
    encoder = BytesIntEncoder(chars.encode())

    def _test_encoding(self, b_in: bytes):
        i = self.encoder.encode(b_in)
        self.assertIsInstance(i, int)
        b_out = self.encoder.decode(i)
        self.assertIsInstance(b_out, bytes)
        self.assertEqual(b_in, b_out)
        # print(b_in, i)

    def test_thoroughly_with_small_str(self):
        for s_len in range(4):
            for s in itertools.combinations_with_replacement(self.chars, s_len):
                s = ''.join(s)
                b_in = s.encode()
                self._test_encoding(b_in)

    def test_randomly_with_large_str(self):
        for s_len in range(256):
            num_samples = {s_len <= 16: 2 ** s_len,
                           16 < s_len <= 32: s_len ** 2,
                           s_len > 32: s_len * 2,
                           s_len > 64: s_len,
                           s_len > 128: 2}[True]
            # print(s_len, num_samples)
            for _ in range(num_samples):
                b_in = ''.join(random.choices(self.chars, k=s_len)).encode()
                self._test_encoding(b_in)


if __name__ == '__main__':
    unittest.main()

Usage example:

>>> encoder = BytesIntEncoder()
>>> s = 'Test123'
>>> b = s.encode()
>>> b
b'Test123'

>>> encoder.encode(b)
3908257788270
>>> encoder.decode(_)
b'Test123'

Answer 4

so I needed transfer a dictionary in terms of numbers, it may look kinda ugly but it's efficient in the way that every char (english letters) is exactly 2 numbers but it's capable of transfering any kind of unicode char

import json

myDict = {
    "le key": "le Valueue",
    2 : {
        "heya": 1234569,
        "3": 4
    },
    'Α α, Β β, Γ γ' : 'שלום'
}
def convertDictToNum(toBeConverted):
    return int(''.join([(lambda c: c if len(c) ==2 else '0'+c )(str(ord(c) - 26)) for c in str(json.dumps(toBeConverted))]))

def loadDictFromNum(toBeDecoded):
    toBeDecoded = str(toBeDecoded)
    return json.loads(''.join([chr(int(toBeDecoded[cut:cut + 2]) + 26) for cut in range(0, len(toBeDecoded), 2)]))

numbersDict = convertDictToNum(myDict)
print(numbersDict)
# 9708827506817595083206088....
recoveredDict = loadDictFromNum(numbersDict)
print(recoveredDict)
# {'le key': 'le Valueue', '2': {'heya': 1234569, '3': 4}, 'Α α, Β β, Γ γ': 'שלום'}

Answer 5

I wanted to do the same and came up with my own algorithm. You can decide it is worth using. You will always get the same int for the same provided input. If you provide string with the same characters but in different order you will get different result.

import hashlib

string="example"
str_as_sha1hash = hashlib.sha1(string.encode()).hexdigest()
result = 0
for idx, char in enumerate(str_as_sha1hash):
    result = result + ord(char) * (idx + 1)
print(result)

You should get 61071 for word ' example '. If you try with ' exampel ' you should receive 55095 . If you think you need stronger hashing algorithm than sha-1 you can replace it with anything available in hashlib library. In the end if you need str instead of int you can do of course do str(result).