Argparse Tutorial

author:

Tshepang Mbambo

This tutorial is intended to be a gentle introduction toargparse, therecommended command-line parsing module in the Python standard library.

Note

The standard library includes two other libraries directly relatedto command-line parameter processing: the lower leveloptparsemodule (which may require more code to configure for a given application,but also allows an application to request behaviors thatargparsedoesn’t support), and the very low levelgetopt (which specificallyserves as an equivalent to thegetopt() family of functionsavailable to C programmers).While neither of those modules is covered directly in this guide, many ofthe core concepts inargparse first originated inoptparse, sosome aspects of this tutorial will also be relevant tooptparse users.

Concepts

Let’s show the sort of functionality that we are going to explore in thisintroductory tutorial by making use of thels command:

$lscpython  devguide  prog.py  pypy  rm-unused-function.patch$lspypyctypes_configure  demo  dotviewer  include  lib_pypy  lib-python ...$ls-ltotal 20drwxr-xr-x 19 wena wena 4096 Feb 18 18:51 cpythondrwxr-xr-x  4 wena wena 4096 Feb  8 12:04 devguide-rwxr-xr-x  1 wena wena  535 Feb 19 00:05 prog.pydrwxr-xr-x 14 wena wena 4096 Feb  7 00:59 pypy-rw-r--r--  1 wena wena  741 Feb 18 01:01 rm-unused-function.patch$ls--helpUsage: ls [OPTION]... [FILE]...List information about the FILEs (the current directory by default).Sort entries alphabetically if none of -cftuvSUX nor --sort is specified....

A few concepts we can learn from the four commands:

  • Thels command is useful when run without any options at all. It defaultsto displaying the contents of the current directory.

  • If we want beyond what it provides by default, we tell it a bit more. Inthis case, we want it to display a different directory,pypy.What we did is specify what is known as a positional argument. It’s named sobecause the program should know what to do with the value, solely based onwhere it appears on the command line. This concept is more relevantto a command likecp, whose most basic usage iscpSRCDEST.The first position iswhat you want copied, and the secondposition iswhere you want it copied to.

  • Now, say we want to change behaviour of the program. In our example,we display more info for each file instead of just showing the file names.The-l in that case is known as an optional argument.

  • That’s a snippet of the help text. It’s very useful in that you cancome across a program you have never used before, and can figure outhow it works simply by reading its help text.

The basics

Let us start with a very simple example which does (almost) nothing:

importargparseparser=argparse.ArgumentParser()parser.parse_args()

Following is a result of running the code:

$pythonprog.py$pythonprog.py--helpusage: prog.py [-h]options:  -h, --help  show this help message and exit$pythonprog.py--verboseusage: prog.py [-h]prog.py: error: unrecognized arguments: --verbose$pythonprog.pyfoousage: prog.py [-h]prog.py: error: unrecognized arguments: foo

Here is what is happening:

  • Running the script without any options results in nothing displayed tostdout. Not so useful.

  • The second one starts to display the usefulness of theargparsemodule. We have done almost nothing, but already we get a nice help message.

  • The--help option, which can also be shortened to-h, is the onlyoption we get for free (i.e. no need to specify it). Specifying anythingelse results in an error. But even then, we do get a useful usage message,also for free.

Introducing Positional arguments

An example:

importargparseparser=argparse.ArgumentParser()parser.add_argument("echo")args=parser.parse_args()print(args.echo)

And running the code:

$pythonprog.pyusage: prog.py [-h] echoprog.py: error: the following arguments are required: echo$pythonprog.py--helpusage: prog.py [-h] echopositional arguments:  echooptions:  -h, --help  show this help message and exit$pythonprog.pyfoofoo

Here is what’s happening:

  • We’ve added theadd_argument() method, which is what we use to specifywhich command-line options the program is willing to accept. In this case,I’ve named itecho so that it’s in line with its function.

  • Calling our program now requires us to specify an option.

  • Theparse_args() method actually returns some data from theoptions specified, in this case,echo.

  • The variable is some form of ‘magic’ thatargparse performs for free(i.e. no need to specify which variable that value is stored in).You will also notice that its name matches the string argument givento the method,echo.

Note however that, although the help display looks nice and all, it currentlyis not as helpful as it can be. For example we see that we gotecho as apositional argument, but we don’t know what it does, other than by guessing orby reading the source code. So, let’s make it a bit more useful:

importargparseparser=argparse.ArgumentParser()parser.add_argument("echo",help="echo the string you use here")args=parser.parse_args()print(args.echo)

And we get:

$pythonprog.py-husage: prog.py [-h] echopositional arguments:  echo        echo the string you use hereoptions:  -h, --help  show this help message and exit

Now, how about doing something even more useful:

importargparseparser=argparse.ArgumentParser()parser.add_argument("square",help="display a square of a given number")args=parser.parse_args()print(args.square**2)

Following is a result of running the code:

$pythonprog.py4Traceback (most recent call last):  File "prog.py", line 5, in <module>    print(args.square**2)TypeError: unsupported operand type(s) for ** or pow(): 'str' and 'int'

That didn’t go so well. That’s becauseargparse treats the options wegive it as strings, unless we tell it otherwise. So, let’s tellargparse to treat that input as an integer:

importargparseparser=argparse.ArgumentParser()parser.add_argument("square",help="display a square of a given number",type=int)args=parser.parse_args()print(args.square**2)

Following is a result of running the code:

$pythonprog.py416$pythonprog.pyfourusage: prog.py [-h] squareprog.py: error: argument square: invalid int value: 'four'

That went well. The program now even helpfully quits on bad illegal inputbefore proceeding.

Introducing Optional arguments

So far we have been playing with positional arguments. Let ushave a look on how to add optional ones:

importargparseparser=argparse.ArgumentParser()parser.add_argument("--verbosity",help="increase output verbosity")args=parser.parse_args()ifargs.verbosity:print("verbosity turned on")

And the output:

$pythonprog.py--verbosity1verbosity turned on$pythonprog.py$pythonprog.py--helpusage: prog.py [-h] [--verbosity VERBOSITY]options:  -h, --help            show this help message and exit  --verbosity VERBOSITY                        increase output verbosity$pythonprog.py--verbosityusage: prog.py [-h] [--verbosity VERBOSITY]prog.py: error: argument --verbosity: expected one argument

Here is what is happening:

  • The program is written so as to display something when--verbosity isspecified and display nothing when not.

  • To show that the option is actually optional, there is no error when runningthe program without it. Note that by default, if an optional argument isn’tused, the relevant variable, in this caseargs.verbosity, isgivenNone as a value, which is the reason it fails the truthtest of theif statement.

  • The help message is a bit different.

  • When using the--verbosity option, one must also specify some value,any value.

The above example accepts arbitrary integer values for--verbosity, but forour simple program, only two values are actually useful,True orFalse.Let’s modify the code accordingly:

importargparseparser=argparse.ArgumentParser()parser.add_argument("--verbose",help="increase output verbosity",action="store_true")args=parser.parse_args()ifargs.verbose:print("verbosity turned on")

And the output:

$pythonprog.py--verboseverbosity turned on$pythonprog.py--verbose1usage: prog.py [-h] [--verbose]prog.py: error: unrecognized arguments: 1$pythonprog.py--helpusage: prog.py [-h] [--verbose]options:  -h, --help  show this help message and exit  --verbose   increase output verbosity

Here is what is happening:

  • The option is now more of a flag than something that requires a value.We even changed the name of the option to match that idea.Note that we now specify a new keyword,action, and give it the value"store_true". This means that, if the option is specified,assign the valueTrue toargs.verbose.Not specifying it impliesFalse.

  • It complains when you specify a value, in true spirit of what flagsactually are.

  • Notice the different help text.

Short options

If you are familiar with command line usage,you will notice that I haven’t yet touched on the topic of shortversions of the options. It’s quite simple:

importargparseparser=argparse.ArgumentParser()parser.add_argument("-v","--verbose",help="increase output verbosity",action="store_true")args=parser.parse_args()ifargs.verbose:print("verbosity turned on")

And here goes:

$pythonprog.py-vverbosity turned on$pythonprog.py--helpusage: prog.py [-h] [-v]options:  -h, --help     show this help message and exit  -v, --verbose  increase output verbosity

Note that the new ability is also reflected in the help text.

Combining Positional and Optional arguments

Our program keeps growing in complexity:

importargparseparser=argparse.ArgumentParser()parser.add_argument("square",type=int,help="display a square of a given number")parser.add_argument("-v","--verbose",action="store_true",help="increase output verbosity")args=parser.parse_args()answer=args.square**2ifargs.verbose:print(f"the square of{args.square} equals{answer}")else:print(answer)

And now the output:

$pythonprog.pyusage: prog.py [-h] [-v] squareprog.py: error: the following arguments are required: square$pythonprog.py416$pythonprog.py4--verbosethe square of 4 equals 16$pythonprog.py--verbose4the square of 4 equals 16

  • We’ve brought back a positional argument, hence the complaint.

  • Note that the order does not matter.

How about we give this program of ours back the ability to havemultiple verbosity values, and actually get to use them:

importargparseparser=argparse.ArgumentParser()parser.add_argument("square",type=int,help="display a square of a given number")parser.add_argument("-v","--verbosity",type=int,help="increase output verbosity")args=parser.parse_args()answer=args.square**2ifargs.verbosity==2:print(f"the square of{args.square} equals{answer}")elifargs.verbosity==1:print(f"{args.square}^2 =={answer}")else:print(answer)

And the output:

$pythonprog.py416$pythonprog.py4-vusage: prog.py [-h] [-v VERBOSITY] squareprog.py: error: argument -v/--verbosity: expected one argument$pythonprog.py4-v14^2 == 16$pythonprog.py4-v2the square of 4 equals 16$pythonprog.py4-v316

These all look good except the last one, which exposes a bug in our program.Let’s fix it by restricting the values the--verbosity option can accept:

importargparseparser=argparse.ArgumentParser()parser.add_argument("square",type=int,help="display a square of a given number")parser.add_argument("-v","--verbosity",type=int,choices=[0,1,2],help="increase output verbosity")args=parser.parse_args()answer=args.square**2ifargs.verbosity==2:print(f"the square of{args.square} equals{answer}")elifargs.verbosity==1:print(f"{args.square}^2 =={answer}")else:print(answer)

And the output:

$pythonprog.py4-v3usage: prog.py [-h] [-v {0,1,2}] squareprog.py: error: argument -v/--verbosity: invalid choice: 3 (choose from 0, 1, 2)$pythonprog.py4-husage: prog.py [-h] [-v {0,1,2}] squarepositional arguments:  square                display a square of a given numberoptions:  -h, --help            show this help message and exit  -v, --verbosity {0,1,2}                        increase output verbosity

Note that the change also reflects both in the error message as well as thehelp string.

Now, let’s use a different approach of playing with verbosity, which is prettycommon. It also matches the way the CPython executable handles its ownverbosity argument (check the output ofpython--help):

importargparseparser=argparse.ArgumentParser()parser.add_argument("square",type=int,help="display the square of a given number")parser.add_argument("-v","--verbosity",action="count",help="increase output verbosity")args=parser.parse_args()answer=args.square**2ifargs.verbosity==2:print(f"the square of{args.square} equals{answer}")elifargs.verbosity==1:print(f"{args.square}^2 =={answer}")else:print(answer)

We have introduced another action, “count”,to count the number of occurrences of specific options.

$pythonprog.py416$pythonprog.py4-v4^2 == 16$pythonprog.py4-vvthe square of 4 equals 16$pythonprog.py4--verbosity--verbositythe square of 4 equals 16$pythonprog.py4-v1usage: prog.py [-h] [-v] squareprog.py: error: unrecognized arguments: 1$pythonprog.py4-husage: prog.py [-h] [-v] squarepositional arguments:  square           display a square of a given numberoptions:  -h, --help       show this help message and exit  -v, --verbosity  increase output verbosity$pythonprog.py4-vvv16

  • Yes, it’s now more of a flag (similar toaction="store_true") in theprevious version of our script. That should explain the complaint.

  • It also behaves similar to “store_true” action.

  • Now here’s a demonstration of what the “count” action gives. You’ve probablyseen this sort of usage before.

  • And if you don’t specify the-v flag, that flag is considered to haveNone value.

  • As should be expected, specifying the long form of the flag, we should getthe same output.

  • Sadly, our help output isn’t very informative on the new ability our scripthas acquired, but that can always be fixed by improving the documentation forour script (e.g. via thehelp keyword argument).

  • That last output exposes a bug in our program.

Let’s fix:

importargparseparser=argparse.ArgumentParser()parser.add_argument("square",type=int,help="display a square of a given number")parser.add_argument("-v","--verbosity",action="count",help="increase output verbosity")args=parser.parse_args()answer=args.square**2# bugfix: replace == with >=ifargs.verbosity>=2:print(f"the square of{args.square} equals{answer}")elifargs.verbosity>=1:print(f"{args.square}^2 =={answer}")else:print(answer)

And this is what it gives:

$pythonprog.py4-vvvthe square of 4 equals 16$pythonprog.py4-vvvvthe square of 4 equals 16$pythonprog.py4Traceback (most recent call last):  File "prog.py", line 11, in <module>    if args.verbosity >= 2:TypeError: '>=' not supported between instances of 'NoneType' and 'int'

  • First output went well, and fixes the bug we had before.That is, we want any value >= 2 to be as verbose as possible.

  • Third output not so good.

Let’s fix that bug:

importargparseparser=argparse.ArgumentParser()parser.add_argument("square",type=int,help="display a square of a given number")parser.add_argument("-v","--verbosity",action="count",default=0,help="increase output verbosity")args=parser.parse_args()answer=args.square**2ifargs.verbosity>=2:print(f"the square of{args.square} equals{answer}")elifargs.verbosity>=1:print(f"{args.square}^2 =={answer}")else:print(answer)

We’ve just introduced yet another keyword,default.We’ve set it to0 in order to make it comparable to the other int values.Remember that by default,if an optional argument isn’t specified,it gets theNone value, and that cannot be compared to an int value(hence theTypeError exception).

And:

$pythonprog.py416

You can go quite far just with what we’ve learned so far,and we have only scratched the surface.Theargparse module is very powerful,and we’ll explore a bit more of it before we end this tutorial.

Getting a little more advanced

What if we wanted to expand our tiny program to perform other powers,not just squares:

importargparseparser=argparse.ArgumentParser()parser.add_argument("x",type=int,help="the base")parser.add_argument("y",type=int,help="the exponent")parser.add_argument("-v","--verbosity",action="count",default=0)args=parser.parse_args()answer=args.x**args.yifargs.verbosity>=2:print(f"{args.x} to the power{args.y} equals{answer}")elifargs.verbosity>=1:print(f"{args.x}^{args.y} =={answer}")else:print(answer)

Output:

$pythonprog.pyusage: prog.py [-h] [-v] x yprog.py: error: the following arguments are required: x, y$pythonprog.py-husage: prog.py [-h] [-v] x ypositional arguments:  x                the base  y                the exponentoptions:  -h, --help       show this help message and exit  -v, --verbosity$pythonprog.py42-v4^2 == 16

Notice that so far we’ve been using verbosity level tochange the textthat gets displayed. The following example instead uses verbosity levelto displaymore text instead:

importargparseparser=argparse.ArgumentParser()parser.add_argument("x",type=int,help="the base")parser.add_argument("y",type=int,help="the exponent")parser.add_argument("-v","--verbosity",action="count",default=0)args=parser.parse_args()answer=args.x**args.yifargs.verbosity>=2:print(f"Running '{__file__}'")ifargs.verbosity>=1:print(f"{args.x}^{args.y} == ",end="")print(answer)

Output:

$pythonprog.py4216$pythonprog.py42-v4^2 == 16$pythonprog.py42-vvRunning 'prog.py'4^2 == 16

Specifying ambiguous arguments

When there is ambiguity in deciding whether an argument is positional or for anargument,-- can be used to tellparse_args() thateverything after that is a positional argument:

>>>parser=argparse.ArgumentParser(prog='PROG')>>>parser.add_argument('-n',nargs='+')>>>parser.add_argument('args',nargs='*')>>># ambiguous, so parse_args assumes it's an option>>>parser.parse_args(['-f'])usage: PROG [-h] [-n N [N ...]] [args ...]PROG: error: unrecognized arguments: -f>>>parser.parse_args(['--','-f'])Namespace(args=['-f'], n=None)>>># ambiguous, so the -n option greedily accepts arguments>>>parser.parse_args(['-n','1','2','3'])Namespace(args=[], n=['1', '2', '3'])>>>parser.parse_args(['-n','1','--','2','3'])Namespace(args=['2', '3'], n=['1'])

Conflicting options

So far, we have been working with two methods of anargparse.ArgumentParser instance. Let’s introduce a third one,add_mutually_exclusive_group(). It allows for us to specify options thatconflict with each other. Let’s also change the rest of the program so thatthe new functionality makes more sense:we’ll introduce the--quiet option,which will be the opposite of the--verbose one:

importargparseparser=argparse.ArgumentParser()group=parser.add_mutually_exclusive_group()group.add_argument("-v","--verbose",action="store_true")group.add_argument("-q","--quiet",action="store_true")parser.add_argument("x",type=int,help="the base")parser.add_argument("y",type=int,help="the exponent")args=parser.parse_args()answer=args.x**args.yifargs.quiet:print(answer)elifargs.verbose:print(f"{args.x} to the power{args.y} equals{answer}")else:print(f"{args.x}^{args.y} =={answer}")

Our program is now simpler, and we’ve lost some functionality for the sake ofdemonstration. Anyways, here’s the output:

$pythonprog.py424^2 == 16$pythonprog.py42-q16$pythonprog.py42-v4 to the power 2 equals 16$pythonprog.py42-vqusage: prog.py [-h] [-v | -q] x yprog.py: error: argument -q/--quiet: not allowed with argument -v/--verbose$pythonprog.py42-v--quietusage: prog.py [-h] [-v | -q] x yprog.py: error: argument -q/--quiet: not allowed with argument -v/--verbose

That should be easy to follow. I’ve added that last output so you can see thesort of flexibility you get, i.e. mixing long form options with short formones.

Before we conclude, you probably want to tell your users the main purpose ofyour program, just in case they don’t know:

importargparseparser=argparse.ArgumentParser(description="calculate X to the power of Y")group=parser.add_mutually_exclusive_group()group.add_argument("-v","--verbose",action="store_true")group.add_argument("-q","--quiet",action="store_true")parser.add_argument("x",type=int,help="the base")parser.add_argument("y",type=int,help="the exponent")args=parser.parse_args()answer=args.x**args.yifargs.quiet:print(answer)elifargs.verbose:print(f"{args.x} to the power{args.y} equals{answer}")else:print(f"{args.x}^{args.y} =={answer}")

Note that slight difference in the usage text. Note the[-v|-q],which tells us that we can either use-v or-q,but not both at the same time:

$pythonprog.py--helpusage: prog.py [-h] [-v | -q] x ycalculate X to the power of Ypositional arguments:  x              the base  y              the exponentoptions:  -h, --help     show this help message and exit  -v, --verbose  -q, --quiet

How to translate the argparse output

The output of theargparse module such as its help text and errormessages are all made translatable using thegettext module. Thisallows applications to easily localize messages produced byargparse. See alsoInternationalizing your programs and modules.

For instance, in thisargparse output:

$pythonprog.py--helpusage: prog.py [-h] [-v | -q] x ycalculate X to the power of Ypositional arguments:  x              the base  y              the exponentoptions:  -h, --help     show this help message and exit  -v, --verbose  -q, --quiet

The stringsusage:,positionalarguments:,options: andshowthishelpmessageandexit are all translatable.

In order to translate these strings, they must first be extractedinto a.po file. For example, usingBabel,run this command:

$pybabelextract-omessages.po/usr/lib/python3.12/argparse.py

This command will extract all translatable strings from theargparsemodule and output them into a file namedmessages.po. This command assumesthat your Python installation is in/usr/lib.

You can find out the location of theargparse module on your systemusing this script:

importargparseprint(argparse.__file__)

Once the messages in the.po file are translated and the translations areinstalled usinggettext,argparse will be able to display thetranslated messages.

To translate your own strings in theargparse output, usegettext.

Custom type converters

Theargparse module allows you to specify custom type converters foryour command-line arguments. This allows you to modify user input before it’sstored in theargparse.Namespace. This can be useful when you need topre-process the input before it is used in your program.

When using a custom type converter, you can use any callable that takes asingle string argument (the argument value) and returns the converted value.However, if you need to handle more complex scenarios, you can use a customaction class with theaction parameter instead.

For example, let’s say you want to handle arguments with different prefixes andprocess them accordingly:

importargparseparser=argparse.ArgumentParser(prefix_chars='-+')parser.add_argument('-a',metavar='<value>',action='append',type=lambdax:('-',x))parser.add_argument('+a',metavar='<value>',action='append',type=lambdax:('+',x))args=parser.parse_args()print(args)

Output:

$pythonprog.py-avalue1+avalue2Namespace(a=[('-', 'value1'), ('+', 'value2')])

In this example, we:

  • Created a parser with custom prefix characters using theprefix_charsparameter.

  • Defined two arguments,-a and+a, which used thetype parameter tocreate custom type converters to store the value in a tuple with the prefix.

Without the custom type converters, the arguments would have treated the-aand+a as the same argument, which would have been undesirable. By using customtype converters, we were able to differentiate between the two arguments.

Conclusion

Theargparse module offers a lot more than shown here.Its docs are quite detailed and thorough, and full of examples.Having gone through this tutorial, you should easily digest themwithout feeling overwhelmed.