Python Logger display information

Question

I'm trying to use the logger library in python with the following code:

import logging
import sys

file_handler = logging.FileHandler(filename=r'\\Folder01\Deployment\Folder\logger\logger.log')
stdout_handler = logging.StreamHandler(sys.stdout)
handlers = [file_handler, stdout_handler]

logging.basicConfig(
    level=logging.DEBUG, 
    format='[%(asctime)s] {%(filename)s:%(lineno)d} %(levelname)s - %(message)s',
    handlers=handlers
)

logging.info("Log Test") 

However, I'm struggling with the below three points.

1. I assume with the sysout configuration, the 'Log Test' string is somewhere in the system. How can I access it?

2. How can I show the timestamp and script from which it was generated? This is how my.log file currently looks like.

3. If I try to delete a.log file, it won't allow me because python has it open (even after executing). For autoclosing other kinds of files, there is the option to do with open(...) as file:... Is there here the same possibility?

Answer 1

I tried to reproduce using your code (only changed path of log file).

1. I assume with the sysout configuration, the 'Log Test' string is somewhere in the system. How can I access it?

The sys.stdout stream is the standard output. If no redirection have occured, it points to the terminal (Windows console) where the script was launched from. In Windows, if the script is launched from a GUI process (pyw.exe for example) stdout is the NUL: special file so everything written there just vanishes.

2. How can I show the timestamp and script from which it was generated?...

I could not reproduce your output. My logger.log file contains:

[2021-03-18 14:19:20,603] {ess2.py:14} INFO - Log Test

3. If I try to delete a.log file, it won't allow me because python has it open...

This is a Windows thing. By default, files opened in one process cannot be deleted or opened in write mode from another process. The only way is to pass special options to the CreateFile function. Little can be done, because the FileHandler keeps its file open all along its life.

But if your performance requirements allows it, it is easy to build a custom Handler that would close the file after every message. You can know whether the overhead of opening and closing the file for every message is acceptable, I cannot.

But here is a possible code (mode and encoding parameters should be implemented):

class SpecialHandler(logging.Handler):
    def __init__(self, filename):
        super().__init__()
        self.filename = filename

    def close(self):
        self.filename = None
        super().close()

    def emit(self, record):
        if self.filename is not None:
            with open(self.filename, 'a') as out:
                _ = out.write(self.format(record) + '\n')

If you use it instead of a normal FileHandler, you will be able to remove the file between 2 messages:

file_handler = SpecialHandler(filename=r'\\Folder01\Deployment\Folder\logger\logger.log')