如何计算法语字符？

Question

I have a function that is splitting all words and checking whether it exceeds 245 characters or no. What I want to do is to check if there are any french characters (char) and count them as 2. For instance letter 'a' is counted as 1 but 'à' should be counted as 2.

def count_chars(content: str) -> str:
    chars = ['à', 'â', 'æ', 'ç', 'é', 'è', 'ê', 'ë', 'î', 'ï', 'ô', 'œ', 'ù', 'û', 'ü', 'ÿ',
             'À', 'Â', 'Æ', 'Ç', 'É', 'È', 'Ê', 'Ë', 'Î', 'Ï', 'Ô', 'Œ', 'Ù', 'Û', 'Ü', 'Ÿ']
    words = content.split()
    new_content = ''
    for word in words:
        if len(new_content + word) <= 245:
            new_content += ' ' + word
        else:
            break
    return new_content.strip()

Answer 1

It's probably best to keep a length variable:

def count_chars(content: str) -> str:
    chars = {'à', 'â', 'æ', 'ç', 'é', 'è', 'ê', 'ë', 'î', 'ï', 'ô', 'œ', 'ù', 'û', 'ü', 'ÿ',
             'À', 'Â', 'Æ', 'Ç', 'É', 'È', 'Ê', 'Ë', 'Î', 'Ï', 'Ô', 'Œ', 'Ù', 'Û', 'Ü', 'Ÿ'}
    words = content.split()
    new_content = ''
    length = 0
    for word in words:
        word_length = sum(2 if i in chars else 1 for char in word)
        if length + word_length <= 245:
            new_content += ' ' + word
            length += word_length
        else:
            break
    return new_content.strip()

Note: I also turned chars into a set for a (theoretical) speed of O(1) .

Answer 2

After throughout analysis and research I found much better way to do it. The function below is caculating bytes based on unicode standards so no need to provide with list of characters, link to documentation where everything is exaplained is here :

def utf8len(s):
    return len(s.encode('utf-8'))


def trunc_string(content: str) -> str:
    # split words to count number of characters
    words = content.split()
    new_content = ''
    length = 0
    for word in words:
        # count all non-standard characters as more than 1
        word_length = utf8len(word)
        # check if number of characters is higher than 245
        if length + word_length <= 245:
            new_content += ' ' + word
            length += word_length
        else:
            break
    return new_content.strip()