UTF-8 encoding issue with logging imodule in python 3

Question

In my python (3.7.4) app I'm trying to analyze a few xml files (encoded with utf-8) in parallel (a few threads) and log a few messages regarding the rows in my log.

my code :

import logging
....
logging.basicConfig(filename='app.log', filemode='w',format='[%(asctime)s] [%(process)d][%(threadName)s - %(thread)d]- %(message)s', level=logging.INFO,encoding='utf-8')
xmldoc = minidom.parse(xml_file)
logging.info("starting to parse file : {}".format(xml_file))
itemsList = xmldoc.getElementsByTagName("Item")
for item in itemsList:
    currentItemName = item.getElementsByTagName("itemName")[0].firstChild.data
    logging.info("parsing item :{}".format(currentItemName))
    ...

the error that I'm getting regarding the logging row :

UnicodeEncodeError: 'charmap' codec can't encode characters in position 81-86: character maps to <undefined>

The file that I'm parsing is an xml file encoded in utf8 :

<?xml version="1.0" encoding="utf-8"?>

stack trace :

--- Logging error ---
Traceback (most recent call last):
  File "C:\Users\myuser\AppData\Local\Programs\Python\Python37-32\lib\logging\__init__.py", line 1028, in emit
    stream.write(msg + self.terminator)
  File "C:\Users\myuser\AppData\Local\Programs\Python\Python37-32\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 82-87: character maps to <undefined>
Call stack:
  File "C:\Users\myuser\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 890, in _bootstrap
    self._bootstrap_ifnner()
  File "C:\Users\myuser\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in _bootstrap_inner
    self.run()
  File "C:\Users\myuser\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\myuser\PycharmProjects\myproj\pars\xmlparsers.py", line 88, in parseXML
    logging.info("parsing item :{}".format(currentItem))
Message: 'parsing item לד500גר'

Also tried(didnt solve my problem) :

logging.basicConfig(
    handlers=[logging.FileHandler('main.log', 'w', 'utf-8')],
    format='[%(asctime)s] [%(process)d][%(threadName)s - %(thread)d]- %(message)s',
    level=logging.INFO,
    encoding='UTF-8')

I will be happy if someone can help me solve this problem..

Answer 1

It seems that once you configure the root logger you can't edit its setting. I configured the basicConfig of the logger in a different file and that's why all the changes in the current file (like enabling encoding for the handler) didn't take effect.