How to get arguments for 2 different objects from the command line when there is a naming conflict in Python argparse

Question

I have two classes, A and B, that each have their own argument parser defined (using argparse) I now want to add functionality to A, so that it calls class B. I am doing this using composition (ie A has an instance of object B)

I asked here how to combine two arg parse objects, so that argparseA will now include the arguments in argparseB in the question Can two Python argparse objects be combined? My problem is as follows: Both A and B have arguments with the same names. BUT- I need two different values to be entered by the user for (ie. argpaseA.val1 needs to get values argparseA.val1 and argParseB.val1)

(The obvious solution is renaming val1 in either argparseA or argpaseB, however there are over 50 scripts that already inherit class A, and 50 scripts that inherit class B, so I want the changes to A and B to be as minimal as possible.)

I thought of adding a new and differently named argument to argpaseA called val2, that can then be passed to argparseB as val1.

My question is- what is the proper way of doing such a conversion or arguments from argparseA to argparseB?

Or is there a better way to design this?

Answer 1

I'm going to guess that you are trying my parents suggestion, and illustrate what might be going on. But even if you've adopted another approach this may help.

import argparse

parserA=argparse.ArgumentParser()
a1=parserA.add_argument('-f','--foo','--bar')
print(a1)
print()

parserB=argparse.ArgumentParser()
b1=parserB.add_argument('-g','--goo','--bar')
print(b1)
b1.dest='bar'    # can change attributes like dest after creation
print()

# parser with parents; not the conflict_handler
parserC=argparse.ArgumentParser(conflict_handler='resolve',
    parents=[parserA, parserB])
print(parserC._actions)   # the actions (arguments) of C
print()
parserA.print_help()
print()
parserC.print_help()   # uses the C._actions

which produces

1445:~/mypy$ python3 stack38071986.py 
_StoreAction(option_strings=['-f', '--foo', '--bar'], dest='foo',      
    nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)

_StoreAction(option_strings=['-g', '--goo', '--bar'], dest='goo', 
    nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)

[_StoreAction(option_strings=['-f', '--foo'], dest='foo', nargs=None, 
    const=None, default=None, type=None, choices=None, help=None, metavar=None), 
 _HelpAction(option_strings=['-h', '--help'], dest='help', nargs=0, 
    const=None, default='==SUPPRESS==', type=None, choices=None, help='show this help message and exit', metavar=None), 
 _StoreAction(option_strings=['-g', '--goo', '--bar'], dest='bar', 
    nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)]

usage: stack38071986.py [-f FOO]

optional arguments:
  help               show this help message and exit
  -f FOO, --foo FOO

usage: stack38071986.py [-f FOO] [-h] [-g BAR]

optional arguments:
  -f FOO, --foo FOO
  -h, --help            show this help message and exit
  -g BAR, --goo BAR, --bar BAR

Normal store behavior is to store the value at the dest attribute in the args namespace, setattr(namespace, action.dest, value) . dest default is the first long option string (minus the --), but may be set as a parameter (for optionals), or after creation.

When one or more option string (flag) of a new action conflicts with an existing action, the conflict_handler is evoked. The default handler raises an error, but here I use a resolve handler. It tries to remove just enough of the existing argument to resolve the conflict.

All 3 parsers create a -h (help) action. resolve in parserC removes all but one. Note that [-h] is missing from the parserA help.

The --bar defined in parserB conflicts with the same string in parserA . resolve has removed it from A's definition, but -f and --foo remain.

The creation of parserC has messed up the other parsers; so I'd recommend only using parserC.parse_args() in this run.

We could write a different conflict_handler method. The resolve one has some rough edges, and doesn't get used that often.

I am using some features that aren't documented. Some consider that to be unsafe. But if you want unusual behavior you have to accept some risks. Plus the argparse documentation is not the last word on its behavior, and is easier to change than the code itself. The developers are almost paranoid about backward conflicts when making changes.

==================

Here's a stab at customizing the resolve conflict handler

import argparse

def pp(adict):
    for k in adict:
        v=adict[k]
        print('Action %10s:'%k,v.option_strings, v.dest)

def new_resolve(self, action, conflicting_actions):
    rename_dict={'--var':'--var1'}
    for option_string, action in conflicting_actions:
        new_string = rename_dict.get(option_string, None)
        if new_string:
            # rename rather than replace
            print(action.option_strings)
            action.option_strings = [new_string]
            action.dest = new_string[2:]

            pp(self._option_string_actions)
            a1=self._option_string_actions.pop(option_string, None)
            print(a1)
            self._option_string_actions[new_string] = a1
            pp(self._option_string_actions)

        else:
            # regular remove action
            action.option_strings.remove(option_string)
            self._option_string_actions.pop(option_string, None)

            # if the option now has no option string, remove it from the
            # container holding it
            if not action.option_strings:
                action.container._remove_action(action)

argparse._ActionsContainer._handle_conflict_resolve=new_resolve

parserA=argparse.ArgumentParser()
a1=parserA.add_argument('-f','--foo')
a1=parserA.add_argument('--var')

parserB=argparse.ArgumentParser()
b1=parserB.add_argument('-g','--goo')
b1=parserB.add_argument('--var')

parserC=argparse.ArgumentParser(conflict_handler='resolve',
   parents=[parserA, parserB],
   add_help=False)

parserA.print_help()
print()
parserC.print_help()
print(parserC.parse_args())

which produces

1027:~/mypy$ python3 stack38071986.py --var1 1 --var 3
['--var']
Action      --var: ['--var1'] var1
Action         -g: ['-g', '--goo'] goo
Action      --foo: ['-f', '--foo'] foo
Action         -h: ['-h', '--help'] help
Action      --goo: ['-g', '--goo'] goo
Action     --help: ['-h', '--help'] help
Action         -f: ['-f', '--foo'] foo
_StoreAction(option_strings=['--var1'], dest='var1', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)
Action         -g: ['-g', '--goo'] goo
Action     --var1: ['--var1'] var1
Action      --foo: ['-f', '--foo'] foo
Action         -h: ['-h', '--help'] help
Action      --goo: ['-g', '--goo'] goo
Action     --help: ['-h', '--help'] help
Action         -f: ['-f', '--foo'] foo

usage: stack38071986.py [-f FOO] [--var1 VAR1]

optional arguments:
  help               show this help message and exit
  -f FOO, --foo FOO
  --var1 VAR1

usage: stack38071986.py [-f FOO] [--var1 VAR1] [-h] [-g GOO] [--var VAR]

optional arguments:
  -f FOO, --foo FOO
  --var1 VAR1
  -h, --help         show this help message and exit
  -g GOO, --goo GOO
  --var VAR

Namespace(foo=None, goo=None, var='3', var1='1')

The conflict handler functions are defined in a super class, so I can't just subclass ArgumentParser to add a new one. It's easy to add custom Action and FormatHandler classes, but this isn't as easy. My guess is that noone has tried this customization.

So my kludge is to write a modification of the resolve method, and link it in - on the fly. Not clean, but enough for testing.

This handler knows the identity of the existing Action ( action argument), but not the new one (ie the --var from ParserA, but not the --var from ParserB). So I am modifying the 'name' of that existing one. For now this method has to know, via its rename_dict the option string that is to be replaced and the new name.

Having done this, I'm thinking that it might be easier to write a custom version of the parents mechanism. One that could be used as:

parserC = argparse.ArgumentParser()
parserC.add_argument(...)   # C's own arguments
copy_arguments(parserC, [parserA, parserB], rename_dict={...})

==========================

I like this better - a custom parents mechanism, one that lets me specify a skip_list and replace_dict . (I may delete the about solution).

import argparse

def add_actions(parser, other, skip_list=None, rename_dict=None):
    # adapted from _add_container_actions (used for parents)
    # copy (by reference) selected actions from other to parser
    # can skip actions (to avoid use of conflict_handler)
    # can rename other actions (again to avoid conflict)
    if skip_list is None:
        skip_list = ['-h','--help']
    if rename_dict is None:
        rename_dict = {}

    # group handling as before
    # collect groups by titles
    title_group_map = {}
    for group in parser._action_groups:
        if group.title in title_group_map:
            msg = _('cannot merge actions - two groups are named %r')
            raise ValueError(msg % (group.title))
        title_group_map[group.title] = group

    # map each action to its group
    group_map = {}
    for group in other._action_groups:

        # if a group with the title exists, use that, otherwise
        # create a new group matching the other's group
        if group.title not in title_group_map:
            title_group_map[group.title] = parser.add_argument_group(
                title=group.title,
                description=group.description,
                conflict_handler=group.conflict_handler)

        # map the actions to their new group
        for action in group._group_actions:
            group_map[action] = title_group_map[group.title]

    # add other's mutually exclusive groups
    # NOTE: if add_mutually_exclusive_group ever gains title= and
    # description= then this code will need to be expanded as above
    for group in other._mutually_exclusive_groups:
        mutex_group = parser.add_mutually_exclusive_group(
            required=group.required)

        # map the actions to their new mutex group
        for action in group._group_actions:
            group_map[action] = mutex_group

    # add all actions to this other or their group

    # addition with skip and rename
    for action in other._actions:
        option_strings = action.option_strings
        if any([s for s in option_strings if s in skip_list]):
             print('skipping ', action.dest)
             continue
        else:
            sl = [s for s in option_strings if s in rename_dict]
            if len(sl):
                 mod = rename_dict[sl[0]]
                 action.dest = action.dest+mod
                 action.option_strings = [option_strings[0]+mod]
        group_map.get(action, parser)._add_action(action)

parserA=argparse.ArgumentParser()
a1=parserA.add_argument('-f','--foo')
a1=parserA.add_argument('--var')

parserB=argparse.ArgumentParser()
b1=parserB.add_argument('-g','--goo')
b1=parserB.add_argument('--var')

parserC=argparse.ArgumentParser()
# parserC.add_argument('baz')
add_actions(parserC, parserA, rename_dict={'--var':'A'})
add_actions(parserC, parserB, rename_dict={'--var':'B'})

parserC.print_help()
print(parserC.parse_args())

and a sample run

2245:~/mypy$ python3 stack38071986_1.py --varA 1 --varB 3
skipping  help
skipping  help

usage: stack38071986_1.py [-h] [-f FOO] [--varA VARA] [-g GOO] [--varB VARB]

optional arguments:
  -h, --help         show this help message and exit
  -f FOO, --foo FOO
  --varA VARA
  -g GOO, --goo GOO
  --varB VARB
Namespace(foo=None, goo=None, varA='1', varB='3')

=============================

If I add

print('parserC actions')
for action in parserC._actions:
    print(action.option_strings, action.dest)

I get this printout

parserC actions
['-h', '--help'] help
['-f', '--foo'] foo
['--varA'] varA
['-g', '--goo'] goo
['--varB'] varB

_actions is the list of Actions (argument) of the parser. It is 'hidden' so use with caution, but I don't anticipate any changes in a fundamental property like this. You can modify many of the attributes of these actions, such as the dest

For example if I rename some of the dest :

for action in parserC._actions:
    if action.dest.startswith('var'):
        action.dest = action.dest+'_C'
print(parserC.parse_args())

the namespace will look like

Namespace(foo=None, goo=None, varA_C=None, varB_C='one')