Python logging creating extra log files

Question

I am trying to use logging to create log files for a program. I'm doing something like this:

if not os.path.exists(r'.\logs'):
        os.mkdir(r'.\logs')

logging.basicConfig(filename = rf'.\logs\log_{time.ctime().replace(":", "-").replace(" ", "_")}.log',
                    format = '%(asctime)s %(name)s %(levelname)s %(message)s',
                    level = logging.DEBUG)

def foo():
        # do stuff ...
        logging.debug('Done some stuff')

        # do extra stuff ...
        logging.debug('Did extra stuff')

        # some parallel map that does NOT use logging in the mapping function
        logging.debug('Done mapping')


if __name__ == '__main__':
        foo()

All goes well ant the log is created with the correct information in it:

logs
    log_Wed_Feb_14_09-23-32_2018.log

Only that at the end, for some reason, it also creates 2 additional log files and leaves them empty:

logs
    log_Wed_Feb_14_09-23-32_2018.log
    log_Wed_Feb_14_09-23-35_2018.log
    log_Wed_Feb_14_09-23-39_2018.log

The timestamps are only a few seconds apart, but all of the logging still only goes in the first log file as it should.

Why is it doing this? Also is there a way to stop it from giving me extra empty files aside from just deleting any empty logs at the end of the program?

Answer 1

Solved. Kind of.

The behaviour using basic config kept happening so I tried to make a custom logger class:

class Logger:
        """Class used to encapsulate logging logic."""
        __slots__ = ['dir',
                     'level',
                     'formatter',
                     'handler',
                     'logger']

        def __init__(self,
                     name: str = '',
                     logdir: str = r'.\logs',
                     lvl: int = logging.INFO,
                     fmt: str = '%(asctime)s %(name)s %(levelname)s %(message)s',
                     hdl: str = rf'.\logs\log_{time.ctime().replace(":", "-").replace(" ", "_")}.log'):

                print('construct')

                if not os.path.exists(logdir):
                        os.mkdir(logdir)

                self.dir = logdir

                self.level = lvl

                self.formatter = logging.Formatter(fmt = fmt)

                self.handler = logging.FileHandler(filename = hdl)
                self.handler.setFormatter(self.formatter)

                self.logger = logging.getLogger(name)
                self.logger.setLevel(self.level)
                self.logger.addHandler(self.handler)

        def log(self, msg: str):
                """Logs the given message to the set level of the logger."""
                self.logger.log(self.level, msg)

        def cleanup(self):
                """Iterates trough the root level of the log folder, removing all log files that have a size of 0."""
                for log_file in (rf'{self.dir}\{log}' for log in next(os.walk(self.dir))[2]
                                 if log.endswith('.log') and os.path.getsize(rf'{self.dir}\{log}') is 0):
                                os.remove(log_file)

        def shutdown(self):
                """Prepares and executes the shutdown and cleanul actions."""
                logging.shutdown()
                self.handler.close()
                self.cleanup()

And tried to pass it as a parameter to functions like this:

def foo(logger = Logger('foo_logger')):

But this approach made it construct a whole new logger each time I called the log method which let again to multiple files. By using one instance of Logger and defaulting the arguments to None I solved the problem of multiple files for this case.

However the initial Basic Config situation remains a mistery.