Important
This PEP is a historical document. The up-to-date, canonical spec,Dependency specifiers, is maintained on thePyPA specs page.
×
See thePyPA specification update process for how to propose changes.
This PEP specifies the language used to describe dependencies for packages.It draws a border at the edge of describing a single dependency - thedifferent sorts of dependencies and when they should be installed is a higherlevel problem. The intent is to provide a building block for higher layerspecifications.
The job of a dependency is to enable tools like pip[1] to find the rightpackage to install. Sometimes this is very loose - just specifying a name, andsometimes very specific - referring to a specific file to install. Sometimesdependencies are only relevant in one platform, or only some versions areacceptable, so the language permits describing all these cases.
The language defined is a compact line based format which is already inwidespread use in pip requirements files, though we do not specify the commandline option handling that those files permit. There is one caveat - theURL reference form, specified inPEP 440 is not actuallyimplemented in pip, but sincePEP 440 is accepted, we use that format ratherthan pip’s current native format.
Any specification in the Python packaging ecosystem that needs to consumelists of dependencies needs to build on an approved PEP for such, butPEP 426 is mostly aspirational - and there are already existingimplementations of the dependency specification which we can instead adopt.The existing implementations are battle proven and user friendly, so adoptingthem is arguably much better than approving an aspirational, unconsumed, format.
All features of the language shown with a name based lookup:
requests[security,tests]>=2.8.1,==2.8.*;python_version<"2.7"
A minimal URL based lookup:
pip@https://github.com/pypa/pip/archive/1.3.1.zip#sha1=da9234ee9982d4bbb3c72346a6de940a148ea686
A dependency specification always specifies a distribution name. It mayinclude extras, which expand the dependencies of the named distribution toenable optional features. The version installed can be controlled usingversion limits, or giving the URL to a specific artifact to install. Finallythe dependency can be made conditional using environment markers.
We first cover the grammar briefly and then drill into the semantics of eachsection later.
A distribution specification is written in ASCII text. We use a parsley[2] grammar to provide a precise grammar. It is expected that thespecification will be embedded into a larger system which offers framing suchas comments, multiple line support via continuations, or other such features.
The full grammar including annotations to build a useful parse tree isincluded at the end of the PEP.
Versions may be specified according to thePEP 440 rules. (Note:URI is defined instd-66):
version_cmp=wsp*'<'|'<='|'!='|'=='|'>='|'>'|'~='|'==='version=wsp*(letterOrDigit|'-'|'_'|'.'|'*'|'+'|'!')+version_one=version_cmpversionwsp*version_many=version_one(wsp*','version_one)*versionspec=('('version_many')')|version_manyurlspec='@'wsp*<URI_reference>
Environment markers allow making a specification only take effect in someenvironments:
marker_op=version_cmp|(wsp*'in')|(wsp*'not'wsp+'in')python_str_c=(wsp|letter|digit|'('|')'|'.'|'{'|'}'|'-'|'_'|'*'|'#'|':'|';'|','|'/'|'?'|'['|']'|'!'|'~'|'`'|'@'|'$'|'%'|'^'|'&'|'='|'+'|'|'|'<'|'>')dquote='"'squote='\\''python_str=(squote(python_str_c|dquote)*squote|dquote(python_str_c|squote)*dquote)env_var=('python_version'|'python_full_version'|'os_name'|'sys_platform'|'platform_release'|'platform_system'|'platform_version'|'platform_machine'|'platform_python_implementation'|'implementation_name'|'implementation_version'|'extra'# ONLY when defined by a containing layer)marker_var=wsp*(env_var|python_str)marker_expr=marker_varmarker_opmarker_var|wsp*'('markerwsp*')'marker_and=marker_exprwsp*'and'marker_expr|marker_exprmarker_or=marker_andwsp*'or'marker_and|marker_andmarker=marker_orquoted_marker=';'wsp*marker
Optional components of a distribution may be specified using the extrasfield:
identifier_end = letterOrDigit | (('-' | '_' | '.' )* letterOrDigit)identifier = letterOrDigit identifier_end*name = identifierextras_list = identifier (wsp* ',' wsp* identifier)*extras = '[' wsp* extras_list? wsp* ']'Giving us a rule for name based requirements:
name_req = name wsp* extras? wsp* versionspec? wsp* quoted_marker?
And a rule for direct reference specifications:
url_req = name wsp* extras? wsp* urlspec wsp+ quoted_marker?
Leading to the unified rule that can specify a dependency.:
specification=wsp*(url_req|name_req)wsp*
Non line-breaking whitespace is mostly optional with no semantic meaning. Thesole exception is detecting the end of a URL requirement.
Python distribution names are currently defined inPEP 345. Namesact as the primary identifier for distributions. They are present in alldependency specifications, and are sufficient to be a specification on theirown. However, PyPI places strict restrictions on names - they must match acase insensitive regex or they won’t be accepted. Accordingly, in this PEP welimit the acceptable values for identifiers to that regex. A full redefinitionof name may take place in a future metadata PEP. The regex (run withre.IGNORECASE) is:
^([A-Z0-9]|[A-Z0-9][A-Z0-9._-]*[A-Z0-9])$
An extra is an optional part of a distribution. Distributions can specify asmany extras as they wish, and each extra results in the declaration ofadditional dependencies of the distributionwhen the extra is used in adependency specification. For instance:
requests[security]
Extras union in the dependencies they define with the dependencies of thedistribution they are attached to. The example above would result in requestsbeing installed, and requests own dependencies, and also any dependencies thatare listed in the “security” extra of requests.
If multiple extras are listed, all the dependencies are unioned together.
SeePEP 440 for more detail on both version numbers and versioncomparisons. Version specifications limit the versions of a distribution thatcan be used. They only apply to distributions looked up by name, rather thanvia a URL. Version comparison are also used in the markers feature. Theoptional brackets around a version are present for compatibility withPEP 345but should not be generated, only accepted.
Environment markers allow a dependency specification to provide a rule thatdescribes when the dependency should be used. For instance, consider a packagethat needs argparse. In Python 2.7 argparse is always present. On older Pythonversions it has to be installed as a dependency. This can be expressed as so:
argparse;python_version<"2.7"
A marker expression evaluates to either True or False. When it evaluates toFalse, the dependency specification should be ignored.
The marker language is inspired by Python itself, chosen for the ability tosafely evaluate it without running arbitrary code that could become a securityvulnerability. Markers were first standardised inPEP 345. This PEPfixes some issues that were observed in the design described inPEP 426.
Comparisons in marker expressions are typed by the comparison operator. The<marker_op> operators that are not in <version_cmp> perform the same as theydo for strings in Python. The <version_cmp> operators use thePEP 440version comparison rules when those are defined (that is when bothsides have a valid version specifier). If there is no definedPEP 440behaviour and the operator exists in Python, then the operator falls back tothe Python behaviour. Otherwise an error should be raised. e.g. the followingwill result in errors:
"dog"~="fred"python_version~="surprise"
User supplied constants are always encoded as strings with either' or" quote marks. Note that backslash escapes are not defined, but existingimplementations do support them. They are not included in thisspecification because they add complexity and there is no observable need forthem today. Similarly we do not define non-ASCII character support: all theruntime variables we are referencing are expected to be ASCII-only.
The variables in the marker grammar such as “os_name” resolve to values lookedup in the Python runtime. With the exception of “extra” all values are definedon all Python versions today - it is an error in the implementation of markersif a value is not defined.
Unknown variables must raise an error rather than resulting in a comparisonthat evaluates to True or False.
Variables whose value cannot be calculated on a given Python implementationshould evaluate to0 for versions, and an empty string for all othervariables.
The “extra” variable is special. It is used by wheels to signal whichspecifications apply to a given extra in the wheelMETADATA file, butsince theMETADATA file is based on a draft version ofPEP 426, there isno current specification for this. Regardless, outside of a context where thisspecial handling is taking place, the “extra” variable should result in anerror like all other unknown variables.
| Marker | Python equivalent | Sample values |
|---|---|---|
os_name | os.name | posix,java |
sys_platform | sys.platform | linux,linux2,darwin,java1.8.0_51 (note that “linux”is from Python3 and “linux2” from Python2) |
platform_machine | platform.machine() | x86_64 |
platform_python_implementation | platform.python_implementation() | CPython,Jython |
platform_release | platform.release() | 3.14.1-x86_64-linode39,14.5.0,1.8.0_51 |
platform_system | platform.system() | Linux,Windows,Java |
platform_version | platform.version() | #1SMPFriApr2513:07:35EDT2014JavaHotSpot(TM)64-BitServerVM,25.51-b03,OracleCorporationDarwinKernelVersion14.5.0:WedJul2902:18:53PDT2015;root:xnu-2782.40.9~2/RELEASE_X86_64 |
python_version | '.'.join(platform.python_version_tuple()[:2]) | 3.4,2.7 |
python_full_version | platform.python_version() | 3.4.0,3.5.0b1 |
implementation_name | sys.implementation.name | cpython |
implementation_version | see definition below | 3.4.0,3.5.0b1 |
extra | An error except when defined by the context interpreting thespecification. | test |
Theimplementation_version marker variable is derived fromsys.implementation.version:
defformat_full_version(info):version='{0.major}.{0.minor}.{0.micro}'.format(info)kind=info.releaselevelifkind!='final':version+=kind[0]+str(info.serial)returnversionifhasattr(sys,'implementation'):implementation_version=format_full_version(sys.implementation.version)else:implementation_version="0"
Most of this PEP is already widely deployed and thus offers no compatibilityconcerns.
There are however a few points where the PEP differs from the deployed base.
Firstly,PEP 440 direct references haven’t actually been deployed in the wild,but they were designed to be compatibly added, and there are no knownobstacles to adding them to pip or other tools that consume the existingdependency metadata in distributions - particularly since they won’t bepermitted to be present in PyPI uploaded distributions anyway.
Secondly,PEP 426 markers which have had some reasonable deployment,particularly in wheels and pip, will handle version comparisons withpython_full_version “2.7.10” differently. Specifically in 426 “2.7.10” isless than “2.7.9”. This backward incompatibility is deliberate. We are alsodefining new operators - “~=” and “===”, and new variables -platform_release,platform_system,implementation_name, andimplementation_version which are not present in older markerimplementations. The variables will error on those implementations. Users ofboth features will need to make a judgement as to when support has becomesufficiently widespread in the ecosystem that using them will not causecompatibility issues.
Thirdly,PEP 345 required brackets around version specifiers. In order toacceptPEP 345 dependency specifications, brackets are accepted, but theyshould not be generated.
In order to move forward with any new PEPs that depend on environment markers,we needed a specification that included them in their modern form. This PEPbrings together all the currently unspecified components into a specifiedform.
The requirement specifier was adopted from the EBNF in the setuptoolspkg_resources documentation, since we wish to avoid depending on a de facto, vsPEP specified, standard.
The complete parsley grammar:
wsp = ' ' | '\t'version_cmp = wsp* <'<=' | '<' | '!=' | '==' | '>=' | '>' | '~=' | '==='>version = wsp* <( letterOrDigit | '-' | '_' | '.' | '*' | '+' | '!' )+>version_one = version_cmp:op version:v wsp* -> (op, v)version_many = version_one:v1 (wsp* ',' version_one)*:v2 -> [v1] + v2versionspec = ('(' version_many:v ')' ->v) | version_manyurlspec = '@' wsp* <URI_reference>marker_op = version_cmp | (wsp* 'in') | (wsp* 'not' wsp+ 'in')python_str_c = (wsp | letter | digit | '(' | ')' | '.' | '{' | '}' | '-' | '_' | '*' | '#' | ':' | ';' | ',' | '/' | '?' | '[' | ']' | '!' | '~' | '`' | '@' | '$' | '%' | '^' | '&' | '=' | '+' | '|' | '<' | '>' )dquote = '"'squote = '\\''python_str = (squote <(python_str_c | dquote)*>:s squote | dquote <(python_str_c | squote)*>:s dquote) -> senv_var = ('python_version' | 'python_full_version' | 'os_name' | 'sys_platform' | 'platform_release' | 'platform_system' | 'platform_version' | 'platform_machine' | 'platform_python_implementation' | 'implementation_name' | 'implementation_version' | 'extra' # ONLY when defined by a containing layer ):varname -> lookup(varname)marker_var = wsp* (env_var | python_str)marker_expr = marker_var:l marker_op:o marker_var:r -> (o, l, r) | wsp* '(' marker:m wsp* ')' -> mmarker_and = marker_expr:l wsp* 'and' marker_expr:r -> ('and', l, r) | marker_expr:m -> mmarker_or = marker_and:l wsp* 'or' marker_and:r -> ('or', l, r) | marker_and:m -> mmarker = marker_orquoted_marker = ';' wsp* markeridentifier_end = letterOrDigit | (('-' | '_' | '.' )* letterOrDigit)identifier = < letterOrDigit identifier_end* >name = identifierextras_list = identifier:i (wsp* ',' wsp* identifier)*:ids -> [i] + idsextras = '[' wsp* extras_list?:e wsp* ']' -> ename_req = (name:n wsp* extras?:e wsp* versionspec?:v wsp* quoted_marker?:m -> (n, e or [], v or [], m))url_req = (name:n wsp* extras?:e wsp* urlspec:v (wsp+ | end) quoted_marker?:m -> (n, e or [], v, m))specification = wsp* ( url_req | name_req ):s wsp* -> s# The result is a tuple - name, list-of-extras,# list-of-version-constraints-or-a-url, marker-ast or NoneURI_reference = <URI | relative_ref>URI = scheme ':' hier_part ('?' query )? ( '#' fragment)?hier_part = ('//' authority path_abempty) | path_absolute | path_rootless | path_emptyabsolute_URI = scheme ':' hier_part ( '?' query )?relative_ref = relative_part ( '?' query )? ( '#' fragment )?relative_part = '//' authority path_abempty | path_absolute | path_noscheme | path_emptyscheme = letter ( letter | digit | '+' | '-' | '.')*authority = ( userinfo '@' )? host ( ':' port )?userinfo = ( unreserved | pct_encoded | sub_delims | ':')*host = IP_literal | IPv4address | reg_nameport = digit*IP_literal = '[' ( IPv6address | IPvFuture) ']'IPvFuture = 'v' hexdig+ '.' ( unreserved | sub_delims | ':')+IPv6address = ( ( h16 ':'){6} ls32 | '::' ( h16 ':'){5} ls32 | ( h16 )? '::' ( h16 ':'){4} ls32 | ( ( h16 ':')? h16 )? '::' ( h16 ':'){3} ls32 | ( ( h16 ':'){0,2} h16 )? '::' ( h16 ':'){2} ls32 | ( ( h16 ':'){0,3} h16 )? '::' h16 ':' ls32 | ( ( h16 ':'){0,4} h16 )? '::' ls32 | ( ( h16 ':'){0,5} h16 )? '::' h16 | ( ( h16 ':'){0,6} h16 )? '::' )h16 = hexdig{1,4}ls32 = ( h16 ':' h16) | IPv4addressIPv4address = dec_octet '.' dec_octet '.' dec_octet '.' dec_octetnz = ~'0' digitdec_octet = ( digit # 0-9 | nz digit # 10-99 | '1' digit{2} # 100-199 | '2' ('0' | '1' | '2' | '3' | '4') digit # 200-249 | '25' ('0' | '1' | '2' | '3' | '4' | '5') )# %250-255reg_name = ( unreserved | pct_encoded | sub_delims)*path = ( path_abempty # begins with '/' or is empty | path_absolute # begins with '/' but not '//' | path_noscheme # begins with a non-colon segment | path_rootless # begins with a segment | path_empty ) # zero characterspath_abempty = ( '/' segment)*path_absolute = '/' ( segment_nz ( '/' segment)* )?path_noscheme = segment_nz_nc ( '/' segment)*path_rootless = segment_nz ( '/' segment)*path_empty = pchar{0}segment = pchar*segment_nz = pchar+segment_nz_nc = ( unreserved | pct_encoded | sub_delims | '@')+ # non-zero-length segment without any colon ':'pchar = unreserved | pct_encoded | sub_delims | ':' | '@'query = ( pchar | '/' | '?')*fragment = ( pchar | '/' | '?')*pct_encoded = '%' hexdigunreserved = letter | digit | '-' | '.' | '_' | '~'reserved = gen_delims | sub_delimsgen_delims = ':' | '/' | '?' | '#' | '(' | ')?' | '@'sub_delims = '!' | '$' | '&' | '\\'' | '(' | ')' | '*' | '+' | ',' | ';' | '='hexdig = digit | 'a' | 'A' | 'b' | 'B' | 'c' | 'C' | 'd' | 'D' | 'e' | 'E' | 'f' | 'F'A test program - if the grammar is in a stringgrammar:
importosimportsysimportplatformfromparsleyimportmakeGrammargrammar=""" wsp ... """tests=["A","A.B-C_D","aa","name","name<=1","name>=3","name>=3,<2","name@http://foo.com","name [fred,bar] @ http://foo.com ; python_version=='2.7'","name[quux, strange];python_version<'2.7' and platform_version=='2'","name; os_name=='a' or os_name=='b'",# Should parse as (a and b) or c"name; os_name=='a' and os_name=='b' or os_name=='c'",# Overriding precedence -> a and (b or c)"name; os_name=='a' and (os_name=='b' or os_name=='c')",# should parse as a or (b and c)"name; os_name=='a' or os_name=='b' and os_name=='c'",# Overriding precedence -> (a or b) and c"name; (os_name=='a' or os_name=='b') and os_name=='c'",]defformat_full_version(info):version='{0.major}.{0.minor}.{0.micro}'.format(info)kind=info.releaselevelifkind!='final':version+=kind[0]+str(info.serial)returnversionifhasattr(sys,'implementation'):implementation_version=format_full_version(sys.implementation.version)implementation_name=sys.implementation.nameelse:implementation_version='0'implementation_name=''bindings={'implementation_name':implementation_name,'implementation_version':implementation_version,'os_name':os.name,'platform_machine':platform.machine(),'platform_python_implementation':platform.python_implementation(),'platform_release':platform.release(),'platform_system':platform.system(),'platform_version':platform.version(),'python_full_version':platform.python_version(),'python_version':'.'.join(platform.python_version_tuple()[:2]),'sys_platform':sys.platform,}compiled=makeGrammar(grammar,{'lookup':bindings.__getitem__})fortestintests:parsed=compiled(test).specification()print("%s ->%s"%(test,parsed))
The following changes were made to this PEP based on feedback after its initialimplementation:
python_version was changed fromplatform.python_version()[:3] to'.'.join(platform.python_version_tuple()[:2]), to accommodate potentialfuture versions of Python with 2-digit major and minor versions(e.g. 3.10).[3]python_version(https://github.com/python/peps/issues/560)This document has been placed in the public domain.
Source:https://github.com/python/peps/blob/main/peps/pep-0508.rst
Last modified:2025-02-20 11:58:35 GMT