Python 3 How to ignore errors when writing UTF-8 to file

Question

I have the following program:

with open(r'C:\s_f.csv', 'w', encoding="utf-8", errors="ignore") as outf:
    with open(r'C:\street.csv', 'r', encoding="utf-8", errors="ignore") as f:
        for line in f:
            out_line = line
            out_line = out_line.replace('"','¬')
            out_line = out_line.replace(',','~')
            outf.write(out_line)

For some reason I am still getting:

File "c:\Program Files\Anaconda3\streets.py", line 5
    SyntaxError: Non-UTF-8 code starting with '\xac' in file c:\Program Files\Anaconda3\streets.py on line 5, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details 

How can I ignore the UTF-8 errors in Python 3?

Answer 1

You have saved your source code as something other than UTF-8, most likely as Latin-1 or Windows Codepage 1252.

Your options are to change encoding used for the source (using your text editor), declare the source code encoding on the first or second line of your source file (as indicated by the error message), or use an ASCII-safe escape sequence.

The latter can be done here by using using a \\xhh or \\uhhhh escape sequence:

out_line = out_line.replace('"','\xAC')  # or `'\u00AC'`

\\xac or \\x00ac (case insensitive) encodes the same character in the Unicode standard, the U+00AC NOT SIGN codepoint . If properly encoded to UTF-8, this would use the C2 AC byte sequence, but your .py file was saved with AC only at that point.

If you do know the encoding used but don't want to change it, add a PEP 263 comment to the start of your file (first or second line at the top):

# coding=cp1252

Your best option is to configure your code editor to save the file as UTF-8, however. That's the default encoding Python 3 will use to read your source code.

This has otherwise nothing to do with writing to the CSV file, Python can't even begin to run your code as it can't read the source properly.

Answer 2

Maybe you can use:

# -*- coding: utf-8 -*-

As first line of your code