Delete special character, specific texts and blank lines in .txt file

Question

I have this .txt file where there is a special character which looks strange('uparrow', see screenshot). How do I remove this and blank lines along with unnecessary repeated header rows after few rows. My attempt -

remove_text = ['Trial Balance - Total Currency', 'Page:', 'Currency:', 'Balance Type:', 'ENTITY Range:', 'Ledger:', 'ENTITY:', '------------', '']


with open('MICnew.txt') as oldfile, open('MICnew.txt', 'w') as newfile:
    for line in oldfile:
        if not any(bad_word in line for bad_word in remove_text):
            newfile.write(line)

with open('MICnew.txt','r+') as file:
    for line in file:
        if not line.isspace():
            file.write(line)

My codes delete few unnecessary text and their lines but does not delete THE special char and blank lines

Answer 1

您可以使用以下命令删除任何非 ascii 字符：

cleaned_string = string_to_clean.encode("ascii", "ignore").decode()

Answer 2

Or, you can use regex to get rid of any unessecary characters.

import re
with open('MIC.txt') as oldfile, open('MICnew.txt', 'w') as newfile:
    for line in oldfile:
        newfile.write(re.sub(r'[^a-zA-Z_0-9\s]','',line))

Answer 3

This is what worked for me -

     open('MICnew.txt', 'w') as newfile:

    for line in oldfile:
        clean_line = re.sub(r'[^\x00-\x7f]', ' ', line.strip('\x0c'))
        if not clean_line.isspace():
            newfile.write(clean_line)

I was able to arrive at this solution by using suggestions from community members in my other post here