how to set config before import in pytest conftest.py?

Question

I have myprj project, and files are as below.

$ tree myprj/
myprj/
├── prj
│   ├── __init__.py
│   ├── config.py
│   ├── my_logger.py
│   ├── my_module.py
│   └── tests
│       ├── __init__.py
│       ├── conftest.py
│       └── unit
│           ├── __init__.py
│           └── test_my_module.py
├── setup.py
└── tox.ini

config.py

LOG_FILE='/path/exists/on/production/prj.log'

my_logger.py

from prj import config
import logging

class MyLogger():
    def __init__(self):
        self.setup_logger()

    def setup_logger(self):
        logFilename = config.LOG_FILE

        file_hdlr = logging.FileHandler(logFilename)

my_module.py

from prj import my_logger

myLogger = my_logger.MyLogger()


def my_method():
    return 1

test_my_module.py

from prj import my_module

def test_my_method():
    assert 1 == my_module.my_method()

setup.py

from setuptools import setup

setup(name="Tox Testing")

tox.ini

[tox]
envlist = py35

[testenv]
deps=pytest
commands=py.test

When I run tox it fails with path not exists for log file.

GLOB sdist-make: /private/tmp/myprj/setup.py
py35 inst-nodeps: /private/tmp/myprj/.tox/dist/Tox Testing-0.0.0.zip
py35 installed: attrs==17.4.0,pluggy==0.6.0,py==1.5.2,pytest==3.3.2,six==1.11.0,Tox-Testing==0.0.0
py35 runtests: PYTHONHASHSEED='2231398989'
py35 runtests: commands[0] | py.test
========================================================================================================= test session starts =========================================================================================================
platform darwin -- Python 3.5.2, pytest-3.3.2, py-1.5.2, pluggy-0.6.0
rootdir: /private/tmp/myprj, inifile:
collected 0 items / 1 errors

=============================================================================================================== ERRORS ================================================================================================================
__________________________________________________________________________________________ ERROR collecting prj/tests/unit/test_my_module.py __________________________________________________________________________________________
prj/tests/unit/test_my_module.py:1: in <module>
    from prj import my_module
prj/my_module.py:3: in <module>
    myLogger = my_logger.MyLogger()
prj/my_logger.py:6: in __init__
    self.setup_logger()
prj/my_logger.py:11: in setup_logger
    file_hdlr = logging.FileHandler(logFilename)
/Users/nile2691/.pyenv/versions/3.5.2/lib/python3.5/logging/__init__.py:1008: in __init__
    StreamHandler.__init__(self, self._open())
/Users/nile2691/.pyenv/versions/3.5.2/lib/python3.5/logging/__init__.py:1037: in _open
    return open(self.baseFilename, self.mode, encoding=self.encoding)
E   FileNotFoundError: [Errno 2] No such file or directory: '/path/exists/on/production/prj.log'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
======================================================================================================= 1 error in 0.36 seconds =======================================================================================================
ERROR: InvocationError: '/private/tmp/myprj/.tox/py35/bin/py.test'
_______________________________________________________________________________________________________________ summary _______________________________________________________________________________________________________________
ERROR:   py35: commands failed

I get that, log file path is not exists in my local system, and it gives error.

I tried to set the LOG_FILE variable in config.py using conftest.py fixture.

conftest.py

import pytest
from prj import config


@pytest.fixture(scope="session", autouse=True)
def set_config():
    config.LOG_FILE = '/tmp/prj.log'

But still it return same error. If I am able to invoke something before start the test execution, like, py.test 's first execution step, then I can set the LOG_FILE and it would not raise error.

Answer 1

In the global scope of my_module.py , you create an instance my_logger.MyLogger() and in MyLogger.__init__ logging handlers are set up.

Your problem is the anti-pattern of setting up logging handlers at import time . Don't do that.

For apps, logging handlers should be configured at runtime (done by main() or whatever). For libraries, logging handlers should not be configured at all (with the exception of possibly adding a NullHandler ) - it is up to the user of your library to decide how they want handlers configured.

Note that pytest does not require that logging handlers are already configured in order to test the records logged and make assertions about log events - the caplog fixture will insert its own handlers in order to capture the logging events, so it can be used even with an unconfigured logging system.