How to exclude DEFAULTs from Python ConfigParser .items()?

Question

I'm using ConfigParser to load in data from a configuration file as follows:

test.conf:

[myfiles]
fileone: %(datadir)s/somefile.foo
filetwo: %(datadir)s/nudderfile.foo

load.py:

import ConfigParser

config = ConfigParser.ConfigParser({'datadir': '/tmp'})
config.read('test.conf')

print config.items('myfiles')
print config.get('myfiles', 'datadir')

Output:

$ python load.py 
[('datadir', '/tmp'), ('filetwo', '/tmp/nudderfile.foo'), ('fileone', '/tmp/somefile.foo')]
/tmp

I'm surprised that the defaults for variable substitution ('datadir', '/tmp') are showing up as part of the . items() and .get() returns, as if they were values in the configuration file. Is this behavior expected? Any work arounds, so that I can simply iterate .items() without getting the default dictionary values in there, but still using the magic interpolation?

Reference: http://docs.python.org/library/configparser.html

Thanks!

Update: It's been pointed out that this is the expected behavior: defaults are just like any other name/value pair in the configuration file. Similarly, the name/value pairs in the configuration file are also available for "magic interpolation", so if I define:

foo: bar
zap: %(foo)snowl

I'll get [... ('zap': 'barnowl')]

That's pretty neat, but I'm still wondering if I can accomplish what I want to accomplish: iterate over the name/value pairs in my configuration files, with interpolation of variables, without the defaults.

My specific scenario is this: I wanted to initialize the config object with something like {basedir: '/foo/bar'} , as the absolute paths to certain files varies by installation. Then I need to pass that config object around to and have various other classes iterate through the files. I don't want every class that reads the configuration to have to know that it was initialized with certain defaults and that it should ignore them, as they are not actual files. Is this possible? Any way to hide the defaults from .item() and .get() but still have interpolation? Thanks!

Answer 1

Generally, I've found the configparser.Configparser class very helpful, but also lacking. Others have, too .

However, it can be subclassed and extended, sometimes nicely, sometimes not so nicely (=very implementation dependent)

Here's a solution for your problem, tested in Python3:

class ConfigParser(configparser.ConfigParser):
    """Can get options() without defaults
    """
    def options(self, section, no_defaults=False, **kwargs):
        if no_defaults:
            try:
                return list(self._sections[section].keys())
            except KeyError:
                raise NoSectionError(section)
        else:
            return super().options(section, **kwargs)

This is one of the bad examples, because it is partially copying the source code of options() . It would be nicer if the configparser base class RawConfigParser would provide an internal getter of options _options(self, section) which would incorporate the exception cast, and a options() which would make use of that. Then, in subclassing we could reuse the _options() .

For Python 2, I believe the only change is the super() call to super(ConfigParser,self) .

You can then use:

print config.options('myfiles', no_defaults=True)

And also use that list to iterate.

Answer 2

Can't you just filter out the defaults?

eg:

filtered_items = [x for x in config.items('myfiles') if x[0] not in config.defaults()]

Answer 3

Try this:

config = ConfigParser.ConfigParser({'blahblahblah': 'blah'})
config.read('test.conf')

The blahblahblah key will also appear in items , not because it's a template in the .ini file, but because you specified it as a default. This is how ConfigParser treats defaults: if it can't find them in the file, it assigns their default values.

So it appears to me you have a simple confusion of concepts here.

Answer 4

下面应该只给你myfiles部分中的键/值对，而不列出DEFAULT 中的那些：

list(set(config.items('myfiles'))-set(config.items('DEFAULT')))

Answer 5

I'm using ConfigParser to load in data from a configuration file as follows:

test.conf:

[myfiles]
fileone: %(datadir)s/somefile.foo
filetwo: %(datadir)s/nudderfile.foo

load.py:

import ConfigParser

config = ConfigParser.ConfigParser({'datadir': '/tmp'})
config.read('test.conf')

print config.items('myfiles')
print config.get('myfiles', 'datadir')

Output:

$ python load.py 
[('datadir', '/tmp'), ('filetwo', '/tmp/nudderfile.foo'), ('fileone', '/tmp/somefile.foo')]
/tmp

I'm surprised that the defaults for variable substitution ('datadir', '/tmp') are showing up as part of the . items() and .get() returns, as if they were values in the configuration file. Is this behavior expected? Any work arounds, so that I can simply iterate .items() without getting the default dictionary values in there, but still using the magic interpolation?

Reference: http://docs.python.org/library/configparser.html

Thanks!

Update: It's been pointed out that this is the expected behavior: defaults are just like any other name/value pair in the configuration file. Similarly, the name/value pairs in the configuration file are also available for "magic interpolation", so if I define:

foo: bar
zap: %(foo)snowl

I'll get [... ('zap': 'barnowl')]

That's pretty neat, but I'm still wondering if I can accomplish what I want to accomplish: iterate over the name/value pairs in my configuration files, with interpolation of variables, without the defaults.

My specific scenario is this: I wanted to initialize the config object with something like {basedir: '/foo/bar'} , as the absolute paths to certain files varies by installation. Then I need to pass that config object around to and have various other classes iterate through the files. I don't want every class that reads the configuration to have to know that it was initialized with certain defaults and that it should ignore them, as they are not actual files. Is this possible? Any way to hide the defaults from .item() and .get() but still have interpolation? Thanks!