Python argparse AssertionError from metavar `[[USER@]HOST:]FILE`

Question

Argparse 1.1 or 1.4 fails with AssertionError - the weird regex that reads the metavar value seems to blow up argparse - Possibly related to Python argparse AssertionError when number of arguments exceeds threshold ?

Is there any alternative way to create or use the metavar [[USER@]HOST:]FILE ?

Test setup:

$ virtualenv-3.5 --always-copy test2
$ ./test2/bin/pip install argparse

Python code test.py:

#!/Users/[username]/Development/test2/bin/python3.5

import os
import sys
print('sys.prefix', sys.prefix)
sys.path.insert(
    0, os.path.join(sys.prefix, 'lib/python3.5/site-packages'))

import argparse
print('argparse', argparse.__version__)

parser = argparse.ArgumentParser()
parser.add_argument(
    'files',
    metavar='[[USER@]HOST:]FILE',
    nargs=argparse.PARSER,
    )
parser.add_argument('-a', '-A', metavar='PTRN', dest='patterns', default=[])
parser.add_argument('-b', '-B', metavar='PTRN', dest='patterns', default=[])
parser.add_argument('-c', '-C', metavar='PTRN', dest='patterns', default=[])
parser.add_argument('-d', '-D', metavar='PTRN', dest='patterns', default=[])
parser.add_argument('-e', '-E', metavar='PTRN', dest='patterns', default=[])
parser.add_argument('-f', '-F', metavar='PTRN', dest='patterns', default=[])
print(parser.parse_args())

Shell output:

$ ./test.py
sys.prefix /Users/[username]/Development/test2/bin/..
argparse 1.4.0
Traceback (most recent call last):
  File "./test.py", line 24, in <module>
    print(parser.parse_args())
  File "/Users/[username]/Development/test2/bin/../lib/python3.5/site-packages/argparse.py", line 1725, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/Users/[username]/Development/test2/bin/../lib/python3.5/site-packages/argparse.py", line 1754, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/Users/[username]/Development/test2/bin/../lib/python3.5/site-packages/argparse.py", line 1971, in _parse_known_args
    self.error(_('too few arguments'))
  File "/Users/[username]/Development/test2/bin/../lib/python3.5/site-packages/argparse.py", line 2391, in error
    self.print_usage(_sys.stderr)
  File "/Users/[username]/Development/test2/bin/../lib/python3.5/site-packages/argparse.py", line 2353, in print_usage
    self._print_message(self.format_usage(), file)
  File "/Users/[username]/Development/test2/bin/../lib/python3.5/site-packages/argparse.py", line 2309, in format_usage
    return formatter.format_help()
  File "/Users/[username]/Development/test2/bin/../lib/python3.5/site-packages/argparse.py", line 306, in format_help
    help = self._root_section.format_help()
  File "/Users/[username]/Development/test2/bin/../lib/python3.5/site-packages/argparse.py", line 236, in format_help
    func(*args)
  File "/Users/[username]/Development/test2/bin/../lib/python3.5/site-packages/argparse.py", line 358, in _format_usage
    assert ' '.join(pos_parts) == pos_usage
AssertionError

Answer 1

The workaround is to create a custom help formatter and do post-processing using a sanitized value. One way to do so can be like so:

class MyHelpFormatter(argparse.HelpFormatter):

    def _format_usage(self, usage, actions, groups, prefix):
        result = super(MyHelpFormatter, self)._format_usage(
            usage, actions, groups, prefix)
        return result.format(user_host_file='[[USER@]HOST]:FILE')

Then construct the parser using the custom formattter

parser = argparse.ArgumentParser(formatter_class=MyHelpFormatter)
parser.add_argument(
    'files',
    metavar='{user_host_file}',
    nargs=argparse.PARSER,
    )

Plugging the rest into the code supplied, something like this is produced

$ python demo.py 
('argparse', '1.1')
usage: demo.py [-h] [-a PTRN] [-b PTRN] [-c PTRN] [-d PTRN] [-e PTRN]
               [-f PTRN]
               [[USER@]HOST]:FILE ...
demo.py: error: too few arguments

Naturally, the exact strategy can be as simple as using the str.replace method in the formatter, or more complicated than the str.format method.

Answer 2

Yes, there's a known bug in the argparse usage formatter. Characters in the metavar like brackets produce this assertion error. I could point out the bug/issue or explain the problem. But the simplest solution is to just change your METAVAR to something simpler. Put the extra information in the help line. Another simple option is to provide a custom usage parameter.

http://bugs.python.org/issue11874

Subclassing the HelpFormatter and replacing one or two methods is also good argparse practice. But that requires digging into the code, and understanding what needs to be replaced. The other answer is a good start. The changes suggest in the issue patch are more complicated because it is trying to be general purpose (including the handling of mutually exclusive groups and usage that can span several lines).