Python: How can I enable use of kwargs when calling from command line? (perhaps with argparse)

Question

suppose I have the module myscript.py; This module is production code, and is called often as %dir%>python myscript.py foo bar .

I want to extend it to take keyword arguments. I know that I can take these arguments using the script below, but unfortunately one would have to call it using

%dir%>python myscript.py main(foo, bar) .

I know that I can use the argparse module, but I'm not sure how to do it.

import sys

def main(foo,bar,**kwargs):
    print 'Called myscript with:'
        print 'foo = %s' % foo
        print 'bar = %s' % bar
        if kwargs:
            for k in kwargs.keys():
                print 'keyword argument : %s' % k + ' = ' + '%s' % kwargs[k]   

if __name__=="__main__":
    exec(''.join(sys.argv[1:]))

Answer 1

@Moon beat me to it with a similar solution, but I'd suggest doing the parsing beforehand and passing in actual kwargs :

import sys

def main(foo, bar, **kwargs):
    print('Called myscript with:')
    print('foo = {}'.format(foo))
    print('bar = {}'.format(bar))
    for k, v in kwargs.items():
        print('keyword argument: {} = {}'.format(k, v))

if __name__=='__main__':
    main(sys.argv[1], # foo
         sys.argv[2], # bar
         **dict(arg.split('=') for arg in sys.argv[3:])) # kwargs

# Example use:
# $ python myscript.py foo bar hello=world 'with spaces'='a value'
# Called myscript with:
# foo = foo
# bar = bar
# keyword argument: hello = world
# keyword argument: with spaces = a value

Answer 2

First, you won't be passing an arbitrary Python expression as an argument. It's brittle and unsafe.

To set up the argument parser, you define the arguments you want, then parse them to produce a Namespace object that contains the information specified by the command line call.

import argparse
p = argparse.ArgumentParser()
p.add_argument('foo')
p.add_argument('bar')
p.add_argument('--add-feature-a', dest='a', action='store_true', default=False)

In your __main__ block, you'll parse the arguments, then pass a dictionary produced from the Namespace to main .

if __name__ == '__main__':
    args = p.parse_args()
    main(**vars(args))

Then you'll call your script with a line like

# foo = "3", bar = "6", a = True
python myscript.py 3 6 --add-feature-a

or

# foo = "moo", bar="7.7", a = False
python myscript.py moo 7.7

There's a lot more you can do with argparse , but this is a simple example for getting the value it produces into main .

Answer 3

If you want to pass in keyword arguments as you would in the main function, key=value , you can do it like so:

import sys

def main(foo, bar, *args):
    print "Called my script with"

    print "foo = %s" % foo
    print "bar = %s" % bar

    for arg in args:
        k = arg.split("=")[0]
        v = arg.split("=")[1]

        print "Keyword argument: %s = %s" % (k, v)


if __name__ == "__main__":
    if len(sys.argv) < 3:
        raise SyntaxError("Insufficient arguments.")
    if len(sys.argv) != 3:
        # If there are keyword arguments
        main(sys.argv[1], sys.argv[2], *sys.argv[3:])
    else:
        # If there are no keyword arguments
        main(sys.argv[1], sys.argv[2])

Some examples:

$> python my_file.py a b x=4
Called my script with
foo = a
bar = b
Keyword argument: x = 4
$> python my_file.py foo bar key=value
Called my script with
foo = foo
bar = bar
Keyword argument: key = value

However, this assumes that the key and value do not have any whitespace between them, key = value will not work.

If you are looking for --argument kinds of keyword arguments, you should use argparse .

Answer 4

in two lines of code I can get args and kwargs that I can manipulate like standard args and kwargs:

import sys


if __name__=='__main__':
  argv=argv[1:] 
  kwargs={kw[0]:kw[1] for kw in [ar.split('=') for ar in argv if ar.find('=')>0]}
  args=[arg for arg in argv if arg.find('=')<0]

  #and you can the use args and kwargs as so:
  if 'reset' in args:
    do_some_functions_with_reset()
  a_device_var=kwargs.get('device',False):
  #or whatever you want to do with args and kwargs

and the result is :

$python test.py reset device=foo format=1 bar
->args=['reset','bar']
->kwargs={'device':'foo','format':'1'}

Answer 5

With a bit of introspection, it's possible to set up ArgumentParser from a function's signature, thus mapping command-line parameters directly to function arguments:

import argparse
import inspect

def myfun(mode, count=1, frobify=False, *files):
    print('Doing %s %d times on %s (%sfrobifying)' % (
        mode, count, files, '' if frobify else 'not '
    ))

def funopt(fun, argv=None):
    parser = argparse.ArgumentParser()

    if hasattr(inspect, 'getfullargspec'):
        spec = inspect.getfullargspec(fun)
    else:
        spec = inspect.getargspec(fun)

    num_defaults = len(spec.defaults) if spec.defaults is not None else 0
    for i in range(len(spec.args)):
        if i < len(spec.args) - num_defaults:
            parser.add_argument(spec.args[i])
        elif spec.defaults[i - len(spec.args)] is False:
            parser.add_argument('--' + spec.args[i], 
                                default=False, action='store_true')
        else:
            default = spec.defaults[i - len(spec.args)]
            parser.add_argument('--' + spec.args[i],
                                default=default,
                                type=type(default))
    if spec.varargs is not None:
            parser.add_argument(spec.varargs,
                                nargs='*')

    kwargs = vars(parser.parse_args(argv))
    args = []
    for arg in spec.args:
        args += [kwargs[arg]]
    if spec.varargs is not None:
        args += kwargs[spec.varargs]

    fun(*args)


funopt(myfun)

The result:

$ python test.py               
usage: test.py [-h] [--count COUNT] [--frobify] mode [files [files ...]]
test.py: error: too few arguments

$ python test.py myaction a b c
Doing myaction 1 times on ('a', 'b', 'c') (not frobifying)

$ python test.py --frobify --count=5 myaction a b c 
Doing myaction 5 times on ('a', 'b', 'c') (frobifying)