Conflicting logging with Python pid package

Question

I've been working on a python daemon with pid , and after initial logs directly to console I wanted to switch to file logging, using python's logging module. This is when I run into the problem.

I have start / stop functions to manage the daemon:

import os
import sys
import time
import signal
import lockfile
import logging
import logging.config

import daemon
from pid import PidFile

from mpmonitor.monitor import MempoolMonitor

# logging.config.fileConfig(fname="logging.conf", disable_existing_loggers=False)
# log = logging.getLogger("mpmonitor")


def start():
    print("Starting Mempool Monitor")

    _pid_file = PidFile(pidname="mpmonitor.pid", piddir=curr_dir)

    with daemon.DaemonContext(stdout=sys.stdout,
                              stderr=sys.stderr,
                              stdin=sys.stdin,
                              pidfile=_pid_file):

        # Start the monitor:
        mpmonitor = MempoolMonitor()
        mpmonitor.run()


def stop():
    print("\n{}\n".format(pid_file))

    try:
        with open(pid_file, "r") as f:
            content = f.read()
        f.close()

    except FileNotFoundError as fnf_err:
        print("WARNING - PID file not found, cannot stop daemon.\n({})".format(pid_file))
        sys.exit()


    print("Stopping Mempool Monitor")
    # log.info("Stopping Mempool Monitor")
    pid = int(content)
    os.kill(pid, signal.SIGTERM)

    sys.exit()

which works as you would expect it to. (Note the logging code is commented.)

Uncommenting the logging code breaks everything and some pretty random stuff happens. The error message (trimmed down, full traceback "looks like spam"):

--- Logging error ---
OSError: [Errno 9] Bad file descriptor
Call stack:
  File "/home/leilerg/.local/lib/python3.6/site-packages/pid/__init__.py", line 77, in setup
    self.logger.debug("%r entering setup", self)
Message: '%r entering setup'
Arguments: (<pid.PidFile object at 0x7fc8faa479e8>,)

--- Logging error ---
OSError: [Errno 9] Bad file descriptor
Call stack:
  File "/home/leilerg/.local/lib/python3.6/site-packages/pid/__init__.py", line 170, in create
    self.logger.debug("%r create pidfile: %s", self, self.filename)
Message: '%r create pidfile: %s'
Arguments: (<pid.PidFile object at 0x7fc8faa479e8>, '/home/leilerg/python/mempoolmon/mpmonitor.pid')

Traceback (most recent call last):
  File "/home/leilerg/.local/lib/python3.6/site-packages/pid/__init__.py", line 136, in inner_check
    pid = int(pid_str)
ValueError: invalid literal for int() with base 10: 'DEBUG - 2020-04-'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/leilerg/.local/lib/python3.6/site-packages/pid/__init__.py", line 139, in inner_check
    raise PidFileUnreadableError(exc)
pid.PidFileUnreadableError: invalid literal for int() with base 10: 'DEBUG - 2020-04-'

--- Logging error ---
Traceback (most recent call last):
OSError: [Errno 9] Bad file descriptor
Call stack:
  File "/home/leilerg/.local/lib/python3.6/site-packages/pid/__init__.py", line 197, in close
    self.logger.debug("%r closing pidfile: %s", self, self.filename)
Message: '%r closing pidfile: %s'
Arguments: (<pid.PidFile object at 0x7fc8faa479e8>, '/home/leilerg/python/mempoolmon/mpmonitor.pid')

The random stuff I was referring to is now the file mpmonitor.pid doesn't contain a PID anymore but some attempted logs/error messages

user@mylaptor:mempoolmon: cat mpmonitor.pid

DEBUG - 2020-04-05 10:52:55,676 - PidFile: <pid.PidFile object at 0x7fc8faa479e8> entering setup
DEBUG - 2020-04-05 10:52:55,678 - PidFile: <pid.PidFile object at 0x7fc8faa479e8> create pidfile: /home/leilerg/python/mempoolmon/mpmonitor.pid
DEBUG - 2020-04-05 10:52:55,678 - PidFile: <pid.PidFile object at 0x7fc8faa479e8> check pidfile: /home/leilerg/python/mempoolmon/mpmonitor.pid
DEBUG - 2020-04-05 10:52:55,678 - PidFile: <pid.PidFile object at 0x7fc8faa479e8> closing pidfile: /home/leilerg/python/mempoolmon/mpmonitor.pid

To me this looks like the pid logfile got confused with the PID file somehow. This is odd as I explicitly set disable_existing_loggers=False .

Any ideas?

If relevant, I'm on the latest Linux Mint. I also posted the question on the pid project GitHub, as I suspect this is a bug.

Answer 1

The problem has been solved on the GitHub page, issue 31 .