Updating of a config file written in python

Question

I'm writing a simulation software which should support reading parameters from a config file or from the command line. It is very important to be able to track what was the configuration of a simulation, I'm committing the config file to a local git repository at the start of simulation.

Now if I have parameters on the command line they have higher priority than the ones in the config file. But I also want to commit them. I guess I could save the python objects of a configured simulation, just before it is started. But it would be more elegant, if I could just update the config file with the command line parameters before committing it.

The reason I write the config file in python is that I have to define some python objects in it. I have something like

import SomeSimulationClass
SIMULATOR = SomeSimulationClass

in my config file and the SIMULATOR can then be swapped easily. If I want to use something like configparser I can't have objects I believe.

Is there any easy way to update a python config file? All variable names in it are already defined, I just want to change the values. The only thing I can think of is parsing the file, comparing strings between the file and the command line parameters ...

Answer 1

You can write whatever you want into a file, and then later, Configparser can read from it using values from your variables. Here is an example on how I used Configparser to read environment from config file.

import os
from ConfigParser import SafeConfigParser

conf_filename = os.getenv("CONFIG_FILE")
src_dir  = os.getenv("CONFIG_DIR")
conf_file = os.path.join(src_dir,conf_filename)
parser = SafeConfigParser()
parser.read(conf_file)
section = env
server = parser.get(section, 'host')
db_port = parser.get(section, 'db_port')
ws_port = parser.get(section, 'ws_port')

and the config file itself:

[PROD]
host=xxx-yyy-15
db_port=1521
ws_port=8280
ora_server=xxx-xxx-xxx.com
sid=XXXXX
userid=xxxx
passwd=xxxx
[STAGE]
host=xxx-yyy-04
db_port=1521
ws_port=8280
ora_server=yyy-yyy-yyy.com
sid=YYYYYY
userid=yyyy
passwd=yyyy

Answer 2

I found a way to do what I want. Some small modifications were necessary to my python config module to allow it to be rewritten with the following script, but it works for my purposes:

    with open('merged_config.py', 'w') as merged_config, \
            open(base_config_module.__file__, 'r') as base_config:
        for line in base_config:
            if 'import' in line:
                # copy imports from bas config
                merged_config.write(line)

        for item in dir(base_config_module):
            if item.startswith("__"):
                # ignore __variables like '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__' ...
                continue
            if item == 'SimulationSteps':
                # ingoring my imports
                continue

            item_val = getattr(base_config_module, item)
            # I had to overwrite the __repr__() method of Enums which I used. Everyting else worked fine.
            merged_config.write('%s = %s\n' % (item, repr(item_val)))