Acceptance

This PEP was approved by Guido on python-dev on February 21, 2010[17].

Abstract

This PEP proposes inclusion of the argparse[1] module in the Pythonstandard library in Python 2.7 and 3.2.

Motivation

The argparse module is a command line parsing library which providesmore functionality than the existing command line parsing modules inthe standard library, getopt[2] and optparse[3]. It includessupport for positional arguments (not just options), subcommands,required options, options syntaxes like “/f” and “+rgb”, zero-or-moreand one-or-more style arguments, and many other features the othertwo lack.

The argparse module is also already a popular third-party replacementfor these modules. It is used in projects like IPython (the ScipyPython shell)[4], is included in Debian testing and unstable[5],and since 2007 has had various requests for its inclusion in thestandard library[6][7][8]. This popularity suggests it may bea valuable addition to the Python libraries.

Why aren’t getopt and optparse enough?

One argument against adding argparse is that there are “already twodifferent option parsing modules in the standard library”[9]. Thefollowing is a list of features provided by argparse but not presentin getopt or optparse:

• While it is true there are twooption parsing libraries, thereare no full command line parsing libraries – both getopt andoptparse support only options and have no support for positionalarguments. The argparse module handles both, and as a result, isable to generate better help messages, avoiding redundancies liketheusage= string usually required by optparse.
• The argparse module values practicality over purity. Thus, argparseallows required options and customization of which characters areused to identify options, while optparse explicitly states “thephrase ‘required option’ is self-contradictory” and that the optionsyntaxes-pf,-file,+f,+rgb,/f and/file“are not supported by optparse, and they never will be”.
• The argparse module allows options to accept a variable number ofarguments usingnargs='?',nargs='*' ornargs='+'. Theoptparse module provides an untested recipe for some part of thisfunctionality[10] but admits that “things get hairy when you wantan option to take a variable number of arguments.”
• The argparse module supports subcommands, where a main commandline parser dispatches to other command line parsers depending onthe command line arguments. This is a common pattern in commandline interfaces, e.g.svnco andsvnup.

Why isn’t the functionality just being added to optparse?

Clearly all the above features offer improvements over what isavailable through optparse. A reasonable question then is why thesefeatures are not simply provided as patches to optparse, instead ofintroducing an entirely new module. In fact, the original developmentof argparse intended to do just that, but because of various fairlyconstraining design decisions of optparse, this wasn’t reallypossible. Some of the problems included:

• The optparse module exposes the internals of its parsing algorithm.In particular,parser.largs andparser.rargs are guaranteedto be available to callbacks[11]. This makes it extremelydifficult to improve the parsing algorithm as was necessary inargparse for proper handling of positional arguments and variablelength arguments. For example,nargs='+' in argparse is matchedusing regular expressions and thus has no notion of things likeparser.largs.
• The optparse extension APIs are extremely complex. For example,just to use a simple custom string-to-object conversion function,you have to subclassOption, hack class attributes, and thenspecify your custom option type to the parser, like this:

  classMyOption(Option):TYPES=Option.TYPES+("mytype",)TYPE_CHECKER=copy(Option.TYPE_CHECKER)TYPE_CHECKER["mytype"]=check_mytypeparser=optparse.OptionParser(option_class=MyOption)parser.add_option("-m",type="mytype")

  For comparison, argparse simply allows conversion functions to beused astype= arguments directly, e.g.:

  parser=argparse.ArgumentParser()parser.add_option("-m",type=check_mytype)

  But given the baroque customization APIs of optparse, it is unclearhow such a feature should interact with those APIs, and it isquite possible that introducing the simple argparse API would breakexisting custom Option code.

• Both optparse and argparse parse command line arguments and assignthem as attributes to an object returned byparse_args.However, the optparse module guarantees that thetake_actionmethod of custom actions will always be passed avalues objectwhich provides anensure_value method[12], while the argparsemodule allows attributes to be assigned to any object, e.g.:

  foo_object=...parser.parse_args(namespace=foo_object)foo_object.some_attribute_parsed_from_command_line

  Modifying optparse to allow any object to be passed in would bedifficult because simply passing thefoo_object around insteadof aValues instance will break existing custom actions thatdepend on theensure_value method.

Because of issues like these, which made it unreasonably difficultfor argparse to stay compatible with the optparse APIs, argparse wasdeveloped as an independent module. Given these issues, merging allthe argparse features into optparse with no backwardsincompatibilities seems unlikely.

Deprecation of optparse

Because all of optparse’s features are available in argparse, theoptparse module will be deprecated. However, because of thewidespread use of optparse, the deprecation strategy contains onlydocumentation changes and warnings that will not be visible bydefault:

• Python 2.7+ and 3.2+ – The following note will be added to theoptparse documentation:

  The optparse module is deprecated and will not be developedfurther; development will continue with the argparse module.

• Python 2.7+ – If the Python 3 compatibility flag,-3, isprovided at the command line, then importing optparse will issue aDeprecationWarning. Otherwise no warnings will be issued.
• Python 3.2+ – Importing optparse will issue aPendingDeprecationWarning, which is not displayed by default.

Note that no removal date is proposed for optparse.

Updates to getopt documentation

The getopt module will not be deprecated. However, its documentationwill be updated to point to argparse in a couple of places. At thetop of the module, the following note will be added:

The getopt module is a parser for command line options whose APIis designed to be familiar to users of the C getopt function.Users who are unfamiliar with the C getopt function or who wouldlike to write less code and get better help and error messagesshould consider using the argparse module instead.

Additionally, after the final getopt example, the following note willbe added:

Note that an equivalent command line interface could be producedwith less code by using the argparse module:

importargparseif__name__=='__main__':parser=argparse.ArgumentParser()parser.add_argument('-o','--output')parser.add_argument('-v',dest='verbose',action='store_true')args=parser.parse_args()# ... do something with args.output ...# ... do something with args.verbose ..

Deferred: string formatting

The argparse module supports Python from 2.3 up through 3.2 and as aresult relies on traditional%(foo)s style string formatting. Ithas been suggested that it might be better to use the new style{foo} string formatting[13]. There was some discussion abouthow best to do this for modules in the standard library[14] andseveral people are developing functions for automatically converting%-formatting to {}-formatting[15][16]. When one of these is addedto the standard library, argparse will use them to support bothformatting styles.

Rejected: getopt compatibility methods

Previously, when this PEP was suggesting the deprecation of getoptas well as optparse, there was some talk of adding a method like:

ArgumentParser.add_getopt_arguments(options[,long_options])

However, this method will not be added for a number of reasons:

• The getopt module is not being deprecated, so there is less need.
• This method would not actually ease the transition for any getoptusers who were already maintaining usage messages, because the APIabove gives no way of adding help messages to the arguments.
• Some users of getopt consider it very important that only a singlefunction call is necessary. The API above does not satisfy thisrequirement because bothArgumentParser() andparse_args()must also be called.

Out of Scope: Various Feature Requests

Several feature requests for argparse were made in the discussion ofthis PEP:

• Support argument defaults from environment variables
• Support argument defaults from configuration files
• Support “foo –help subcommand” in addition to the currentlysupported “foo subcommand –help”

These are all reasonable feature requests for the argparse module,but are out of the scope of this PEP, and have been redirected tothe argparse issue tracker.

Discussion: sys.stderr and sys.exit

There were some concerns that argparse by default always writes tosys.stderr and always callssys.exit when invalid argumentsare provided. This is the desired behavior for the vast majority ofargparse use cases which revolve around simple command lineinterfaces. However, in some cases, it may be desirable to keepargparse from exiting, or to have it write its messages to somethingother thansys.stderr. These use cases can be supported bysubclassingArgumentParser and overriding theexit or_print_message methods. The latter is an undocumentedimplementation detail, but could be officially exposed if this turnsout to be a common need.

References

Copyright

This document has been placed in the public domain.