BasicWorkingSet Methods¶

The following methods ofWorkingSet objects are also available asmodule-level functions inpkg_resources that apply to the defaultworking_set instance. Thus, you can use e.g.pkg_resources.require()as an abbreviation forpkg_resources.working_set.require():

require(*requirements)

Ensure that distributions matchingrequirements are activated

requirements must be a string or a (possibly-nested) sequencethereof, specifying the distributions and versions required. Thereturn value is a sequence of the distributions that needed to beactivated to fulfill the requirements; all relevant distributions areincluded, even if they were already activated in this working set.

For the syntax of requirement specifiers, see the section below onRequirements Parsing.

In general, it should not be necessary for you to call this methoddirectly. It’s intended more for use in quick-and-dirty scripting andinteractive interpreter hacking than for production use. If you’re creatingan actual library or application, it’s strongly recommended that you createa “setup.py” script usingsetuptools, and declare all your requirementsthere. That way, tools like pip can automatically detect what requirementsyour package has, and deal with them accordingly.

Note that callingrequire('SomePackage') will not installSomePackage if it isn’t already present. If you need to do this, youshould use theresolve() method instead, which allows you to pass aninstaller callback that will be invoked when a needed distributioncan’t be found on the local machine. You can then have this callbackdisplay a dialog, automatically download the needed distribution, orwhatever else is appropriate for your application. See the documentationbelow on theresolve() method for more information, and also on theobtain() method ofEnvironment objects.

run_script(requires,script_name)

Locate distribution specified byrequires and run itsscript_namescript.requires must be a string containing a requirement specifier.(SeeRequirements Parsing below for the syntax.)

The script, if found, will be executed inthe caller’s globals. That’sbecause this method is intended to be called from wrapper scripts thatact as a proxy for the “real” scripts in a distribution. A wrapper scriptusually doesn’t need to do anything but invoke this function with thecorrect arguments.

If you need more control over the script execution environment, youprobably want to use therun_script() method of aDistributionobject’sMetadata API instead.

iter_entry_points(group,name=None)

Yield entry point objects fromgroup matchingname

Ifname is None, yields all entry points ingroup from alldistributions in the working set, otherwise only ones matching bothgroup andname are yielded. Entry points are yielded from the activedistributions in the order that the distributions appear in the workingset. (For the globalworking_set, this should be the same as the orderthat they are listed insys.path.) Note that within the entry pointsadvertised by an individual distribution, there is no particular ordering.

Please see the section below onEntry Points for more information.

WorkingSet Methods and Attributes¶

These methods are used to query or manipulate the contents of a specificworking set, so they must be explicitly invoked on a particularWorkingSetinstance:

add_entry(entry)

Add a path item to theentries, finding any distributions on it. Youshould use this when you add additional items tosys.path and you wantthe globalworking_set to reflect the change. This method is alsocalled by theWorkingSet() constructor during initialization.

This method usesfind_distributions(entry,True) to find distributionscorresponding to the path entry, and thenadd() them.entry isalways appended to theentries attribute, even if it is alreadypresent, however. (This is becausesys.path can contain the same valuemore than once, and theentries attribute should be able to reflectthis.)

__contains__(dist)

True ifdist is active in thisWorkingSet. Note that only onedistribution for a given project can be active in a givenWorkingSet.

__iter__()

Yield distributions for non-duplicate projects in the working set.The yield order is the order in which the items’ path entries wereadded to the working set.

find(req)

Find a distribution matchingreq (aRequirement instance).If there is an active distribution for the requested project, thisreturns it, as long as it meets the version requirement specified byreq. But, if there is an active distribution for the project and itdoesnot meet thereq requirement,VersionConflict is raised.If there is no active distribution for the requested project,Noneis returned.

resolve(requirements,env=None,installer=None)

List all distributions needed to (recursively) meetrequirements

requirements must be a sequence ofRequirement objects.env,if supplied, should be anEnvironment instance. Ifnot supplied, anEnvironment is created from the working set’sentries.installer, if supplied, will be invoked with eachrequirement that cannot be met by an already-installed distribution; itshould return aDistribution orNone. (See theobtain() methodofEnvironment Objects, below, for more information on theinstallerargument.)

add(dist,entry=None)

Adddist to working set, associated withentry

Ifentry is unspecified, it defaults todist.location. On exit fromthis routine,entry is added to the end of the working set’s.entries(if it wasn’t already present).

dist is only added to the working set if it’s for a project thatdoesn’t already have a distribution active in the set. If it’ssuccessfully added, any callbacks registered with thesubscribe()method will be called. (SeeReceiving Change Notifications, below.)

Note:add() is automatically called for you by therequire()method, so you don’t normally need to use this method directly.

entries

This attribute represents a “shadow”sys.path, primarily useful fordebugging. If you are experiencing import problems, you should checkthe globalworking_set object’sentries againstsys.path, toensure that they match. If they do not, then some part of your programis manipulatingsys.path without updating theworking_setaccordingly. IMPORTANT NOTE: do not directly manipulate this attribute!Setting it equal tosys.path will not fix your problem, any more thanputting black tape over an “engine warning” light will fix your car! Ifthis attribute is out of sync withsys.path, it’s merely anindicatorof the problem, not the cause of it.

Receiving Change Notifications¶

Extensible applications and frameworks may need to receive notification whena new distribution (such as a plug-in component) has been added to a workingset. This is what thesubscribe() method andadd_activation_listener()function are for.

subscribe(callback)

Invokecallback(distribution) once for each active distribution that isin the set now, or gets added later. Because the callback is invoked foralready-active distributions, you do not need to loop over the working setyourself to deal with the existing items; just register the callback andbe prepared for the fact that it will be called immediately by this method.

Note that callbacksmust not allow exceptions to propagate, or they willinterfere with the operation of other callbacks and possibly result in aninconsistent working set state. Callbacks should use a try/except blockto ignore, log, or otherwise process any errors, especially since the codethat caused the callback to be invoked is unlikely to be able to handlethe errors any better than the callback itself.

pkg_resources.add_activation_listener() is an alternate spelling ofpkg_resources.working_set.subscribe().

Locating Plugins¶

Extensible applications will sometimes have a “plugin directory” or a set ofplugin directories, from which they want to load entry points or othermetadata. Thefind_plugins() method allows you to do this, by scanning anenvironment for the newest version of each project that can be safely loadedwithout conflicts or missing requirements.

find_plugins(plugin_env,full_env=None,fallback=True)

Scanplugin_env and identify which distributions could be added to thisworking set without version conflicts or missing requirements.

Example usage:

distributions,errors=working_set.find_plugins(Environment(plugin_dirlist))map(working_set.add,distributions)# add plugins+libs to sys.pathprint"Couldn't load",errors# display errors

Theplugin_env should be anEnvironment instance that contains onlydistributions that are in the project’s “plugin directory” or directories.Thefull_env, if supplied, should be anEnvironment instance thatcontains all currently-available distributions.

Iffull_env is not supplied, one is created automatically from theWorkingSet this method is called on, which will typically mean thatevery directory onsys.path will be scanned for distributions.

This method returns a 2-tuple: (distributions,error_info), wheredistributions is a list of the distributions found inplugin_env thatwere loadable, along with any other distributions that are needed to resolvetheir dependencies.error_info is a dictionary mapping unloadable plugindistributions to an exception instance describing the error that occurred.Usually this will be aDistributionNotFound orVersionConflictinstance.

Most applications will use this method mainly on the masterworking_setinstance inpkg_resources, and then immediately add the returneddistributions to the working set so that they are available on sys.path.This will make it possible to find any entry points, and allow any othermetadata tracking and hooks to be activated.

The resolution algorithm used byfind_plugins() is as follows. First,the project names of the distributions present inplugin_env are sorted.Then, each project’s eggs are tried in descending version order (i.e.,newest version first).

An attempt is made to resolve each egg’s dependencies. If the attempt issuccessful, the egg and its dependencies are added to the output list and toa temporary copy of the working set. The resolution process continues withthe next project name, and no older eggs for that project are tried.

If the resolution attempt fails, however, the error is added to the errordictionary. If thefallback flag is true, the next older version of theplugin is tried, until a working version is found. If false, the resolutionprocess continues with the next plugin project name.

Some applications may have stricter fallback requirements than others. Forexample, an application that has a database schema or persistent objectsmay not be able to safely downgrade a version of a package. Others may wantto ensure that a new plugin configuration is either 100% good or elserevert to a known-good configuration. (That is, they may wish to revert toa known configuration if theerror_info return value is non-empty.)

Note that this algorithm gives precedence to satisfying the dependencies ofalphabetically prior project names in case of version conflicts. If twoprojects named “AaronsPlugin” and “ZekesPlugin” both need different versionsof “TomsLibrary”, then “AaronsPlugin” will win and “ZekesPlugin” will bedisabled due to version conflict.

Requirements Parsing¶

parse_requirements(s)

YieldRequirement objects for a string or iterable of lines. Eachrequirement must start on a new line. See below for syntax.

Requirement.parse(s)

Create aRequirement object from a string or iterable of lines. AValueError is raised if the string or lines do not contain a validrequirement specifier, or if they contain more than one specifier. (Toparse multiple specifiers from a string or iterable of strings, useparse_requirements() instead.)

The syntax of a requirement specifier is defined in full in PEP 508.

Some examples of valid requirement specifiers:

FooProject>=1.2Fizzy[foo,bar]PickyThing>1.6,<=1.9,!=1.8.6SomethingWhoseVersionIDontCareAboutSomethingWithMarker[foo]>1.0;python_version<"2.7"

The project name is the only required portion of a requirement string, andif it’s the only thing supplied, the requirement will accept any versionof that project.

The “extras” in a requirement are used to request optional features of aproject, that may require additional project distributions in order tofunction. For example, if the hypothetical “Report-O-Rama” project offeredoptional PDF support, it might require an additional library in order toprovide that support. Thus, a project needing Report-O-Rama’s PDF featurescould use a requirement ofReport-O-Rama[PDF] to request installationor activation of both Report-O-Rama and any libraries it needs in order toprovide PDF support. For example, you could use:

pipinstallReport-O-Rama[PDF]

To install the necessary packages using pip, or callpkg_resources.require('Report-O-Rama[PDF]') to add the necessarydistributions to sys.path at runtime.

The “markers” in a requirement are used to specify when a requirementshould be installed – the requirement will be installed if the markerevaluates as true in the current environment. For example, specifyingargparse;python_version<"3.0" will not install in an Python 3environment, but will in a Python 2 environment.

Requirement Methods and Attributes¶

__contains__(dist_or_version)

Return true ifdist_or_version fits the criteria for this requirement.Ifdist_or_version is aDistribution object, its project name mustmatch the requirement’s project name, and its version must meet therequirement’s version criteria. Ifdist_or_version is a string, it isparsed using theparse_version() utility function. Otherwise, it isassumed to be an already-parsed version.

TheRequirement object’s version specifiers (.specs) are internallysorted into ascending version order, and used to establish what ranges ofversions are acceptable. Adjacent redundant conditions are effectivelyconsolidated (e.g.">1,>2" produces the same results as">2", and"<2,<3" produces the same results as"<2")."!=" versions areexcised from the ranges they fall within. The version being tested foracceptability is then checked for membership in the resulting ranges.

__eq__(other_requirement)

A requirement compares equal to another requirement if they havecase-insensitively equal project names, version specifiers, and “extras”.(The order that extras and version specifiers are in is also ignored.)Equal requirements also have equal hashes, so that requirements can beused in sets or as dictionary keys.

__str__()

The string form of aRequirement is a string that, if passed toRequirement.parse(), would return an equalRequirement object.

project_name

The name of the required project

key

An all-lowercase version of theproject_name, useful for comparisonor indexing.

extras

A tuple of names of “extras” that this requirement calls for. (These willbe all-lowercase and normalized using thesafe_extra() parsing utilityfunction, so they may not exactly equal the extras the requirement wascreated with.)

specs

A list of(op,version) tuples, sorted in ascending parsed-versionorder. Theop in each tuple is a comparison operator, represented asa string. Theversion is the (unparsed) version number.

marker

An instance ofpackaging.markers.Marker that allows evaluationagainst the current environment. May be None if no marker specified.

url

The location to download the requirement from if specified.

Convenience API¶

In the following functions, thedist argument can be aDistributioninstance, aRequirement instance, or a string specifying a requirement(i.e. project name, version, etc.). If the argument is a string orRequirement, the specified distribution is located (and added to sys.pathif not already present). An error will be raised if a matching distribution isnot available.

Thegroup argument should be a string containing a dotted identifier,identifying an entry point group. If you are defining an entry point group,you should include some portion of your package’s name in the group name so asto avoid collision with other packages’ entry point groups.

load_entry_point(dist,group,name)

Load the named entry point from the specified distribution, or raiseImportError.

get_entry_info(dist,group,name)

Return anEntryPoint object for the givengroup andname fromthe specified distribution. ReturnsNone if the distribution has notadvertised a matching entry point.

get_entry_map(dist,group=None)

Return the distribution’s entry point map forgroup, or the full entrymap for the distribution. This function always returns a dictionary,even if the distribution advertises no entry points. Ifgroup is given,the dictionary maps entry point names to the correspondingEntryPointobject. Ifgroup is None, the dictionary maps group names todictionaries that then map entry point names to the correspondingEntryPoint instance in that group.

iter_entry_points(group,name=None)

Yield entry point objects fromgroup matchingname.

Ifname is None, yields all entry points ingroup from alldistributions in the working set on sys.path, otherwise only ones matchingbothgroup andname are yielded. Entry points are yielded fromthe active distributions in the order that the distributions appear onsys.path. (Within entry points for a particular distribution, however,there is no particular ordering.)

(This API is actually a method of the globalworking_set object; seethe section above onBasic WorkingSet Methods for more information.)

Creating and Parsing¶

EntryPoint(name,module_name,attrs=(),extras=(),dist=None)

Create anEntryPoint instance.name is the entry point name. Themodule_name is the (dotted) name of the module containing the advertisedobject.attrs is an optional tuple of names to look up from themodule to obtain the advertised object. For example, anattrs of("foo","bar") and amodule_name of"baz" would mean that theadvertised object could be obtained by the following code:

importbazadvertised_object=baz.foo.bar

Theextras are an optional tuple of “extra feature” names that thedistribution needs in order to provide this entry point. When theentry point is loaded, these extra features are looked up in thedistargument to find out what other distributions may need to be activatedon sys.path; see theload() method for more details. Theextrasargument is only meaningful ifdist is specified.dist must beaDistribution instance.

EntryPoint.parse(src,dist=None) (classmethod)

Parse a single entry point from stringsrc

Entry point syntax follows the form:

name=some.module:some.attr[extra1,extra2]

The entry name and module name are required, but the:attrs and[extras] parts are optional, as is the whitespace shown betweensome of the items. Thedist argument is passed through to theEntryPoint() constructor, along with the other values parsed fromsrc.

EntryPoint.parse_group(group,lines,dist=None) (classmethod)

Parselines (a string or sequence of lines) to create a dictionarymapping entry point names toEntryPoint objects.ValueError israised if entry point names are duplicated, ifgroup is not a validentry point group name, or if there are any syntax errors. (Note: thegroup parameter is used only for validation and to create moreinformative error messages.) Ifdist is provided, it will be used toset thedist attribute of the createdEntryPoint objects.

EntryPoint.parse_map(data,dist=None) (classmethod)

Parsedata into a dictionary mapping group names to dictionaries mappingentry point names toEntryPoint objects. Ifdata is a dictionary,then the keys are used as group names and the values are passed toparse_group() as thelines argument. Ifdata is a string orsequence of lines, it is first split into .ini-style sections (usingthesplit_sections() utility function) and the section names are usedas group names. In either case, thedist argument is passed through toparse_group() so that the entry points will be linked to the specifieddistribution.

EntryPoint Objects¶

For simple introspection,EntryPoint objects have attributes thatcorrespond exactly to the constructor argument names:name,module_name,attrs,extras, anddist are all available. Inaddition, the following methods are provided:

load()

Load the entry point, returning the advertised Python object. Effectivelycallsself.require() then returnsself.resolve().

require(env=None,installer=None)

Ensure that any “extras” needed by the entry point are available onsys.path.UnknownExtra is raised if theEntryPoint hasextras,but nodist, or if the named extras are not defined by thedistribution. Ifenv is supplied, it must be anEnvironment, and itwill be used to search for needed distributions if they are not alreadypresent on sys.path. Ifinstaller is supplied, it must be a callabletaking aRequirement instance and returning a matching importableDistribution instance or None.

resolve()

Resolve the entry point from its module and attrs, returning the advertisedPython object. RaisesImportError if it cannot be obtained.

__str__()

The string form of anEntryPoint is a string that could be passed toEntryPoint.parse() to produce an equivalentEntryPoint.

Getting or Creating Distributions¶

Most commonly, you’ll obtainDistribution objects from aWorkingSet oranEnvironment. (See the sections above onWorkingSet Objects andEnvironment Objects, which are containers for active distributions andavailable distributions, respectively.) You can also obtainDistributionobjects from one of these high-level APIs:

find_distributions(path_item,only=False)

Yield distributions accessible viapath_item. Ifonly is true, yieldonly distributions whoselocation is equal topath_item. In otherwords, ifonly is true, this yields any distributions that would beimportable ifpath_item were onsys.path. Ifonly is false, thisalso yields distributions that are “in” or “under”path_item, but wouldnot be importable unless their locations were also added tosys.path.

get_distribution(dist_spec)

Return aDistribution object for a givenRequirement or string.Ifdist_spec is already aDistribution instance, it is returned.If it is aRequirement object or a string that can be parsed into one,it is used to locate and activate a matching distribution, which is thenreturned.

However, if you’re creating specialized tools for working with distributions,or creating a new distribution format, you may also need to createDistribution objects directly, using one of the three constructors below.

These constructors all take an optionalmetadata argument, which is used toaccess any resources or metadata associated with the distribution.metadatamust be an object that implements theIResourceProvider interface, or None.If it is None, anEmptyProvider is used instead.Distribution objectsimplement both theIResourceProvider andIMetadataProvider Methods bydelegating them to themetadata object.

Distribution.from_location(location,basename,metadata=None,**kw) (classmethod)

Create a distribution forlocation, which must be a string such as aURL, filename, or other string that might be used onsys.path.basename is a string naming the distribution, likeFoo-1.2-py2.4.egg.Ifbasename ends with.egg, then the project’s name, version, pythonversion and platform are extracted from the filename and used to set thoseproperties of the created distribution. Any additional keyword argumentsare forwarded to theDistribution() constructor.

Distribution.from_filename(filename,metadata=None**kw) (classmethod)

Create a distribution by parsing a local filename. This is a shorter wayof sayingDistribution.from_location(normalize_path(filename),os.path.basename(filename),metadata). In other words, it creates adistribution whose location is the normalize form of the filename, parsingname and version information from the base portion of the filename. Anyadditional keyword arguments are forwarded to theDistribution()constructor.

Distribution(location,metadata,project_name,version,py_version,platform,precedence)

Create a distribution by setting its properties. All arguments areoptional and default to None, except forpy_version (which defaults tothe current Python version) andprecedence (which defaults toEGG_DIST; for more details seeprecedence underDistributionAttributes below). Note that it’s usually easier to use thefrom_filename() orfrom_location() constructors than to specifyall these arguments individually.

Distribution Attributes¶

location

A string indicating the distribution’s location. For an importabledistribution, this is the string that would be added tosys.path tomake it actively importable. For non-importable distributions, this issimply a filename, URL, or other way of locating the distribution.

project_name

A string, naming the project that this distribution is for. Project namesare defined by a project’s setup script, and they are used to identifyprojects on PyPI. When aDistribution is constructed, theproject_name argument is passed through thesafe_name() utilityfunction to filter out any unacceptable characters.

key

dist.key is short fordist.project_name.lower(). It’s used forcase-insensitive comparison and indexing of distributions by project name.

extras

A list of strings, giving the names of extra features defined by theproject’s dependency list (theextras_require argument specified inthe project’s setup script).

version

A string denoting what release of the project this distribution contains.When aDistribution is constructed, theversion argument is passedthrough thesafe_version() utility function to filter out anyunacceptable characters. If noversion is specified at constructiontime, then attempting to access this attribute later will cause theDistribution to try to discover its version by reading itsPKG-INFOmetadata file. IfPKG-INFO is unavailable or can’t be parsed,ValueError is raised.

parsed_version

Theparsed_version is an object representing a “parsed” form of thedistribution’sversion.dist.parsed_version is a shortcut forcallingparse_version(dist.version). It is used to compare or sortdistributions by version. (See theParsing Utilities section below formore information on theparse_version() function.) Note that accessingparsed_version may result in aValueError if theDistributionwas constructed without aversion and withoutmetadata capable ofsupplying the missing version info.

py_version

The major/minor Python version the distribution supports, as a string.For example, “2.7” or “3.4”. The default is the current version of Python.

platform

A string representing the platform the distribution is intended for, orNone if the distribution is “pure Python” and therefore cross-platform.SeePlatform Utilities below for more information on platform strings.

precedence

A distribution’sprecedence is used to determine the relative order oftwo distributions that have the sameproject_name andparsed_version. The default precedence ispkg_resources.EGG_DIST,which is the highest (i.e. most preferred) precedence. The full listof predefined precedences, from most preferred to least preferred, is:EGG_DIST,BINARY_DIST,SOURCE_DIST,CHECKOUT_DIST, andDEVELOP_DIST. Normally, precedences other thanEGG_DIST are usedonly by thesetuptools.package_index module, when sorting distributionsfound in a package index to determine their suitability for installation.“System” and “Development” eggs (i.e., ones that use the.egg-infoformat), however, are automatically given a precedence ofDEVELOP_DIST.

Distribution Methods¶

activate(path=None)

Ensure distribution is importable onpath. Ifpath is None,sys.path is used instead. This ensures that the distribution’slocation is in thepath list, and it also performs any necessarynamespace package fixups or declarations. (That is, if the distributioncontains namespace packages, this method ensures that they are declared,and that the distribution’s contents for those namespace packages aremerged with the contents provided by any other active distributions. Seethe section above onNamespace Package Support for more information.)

pkg_resources adds a notification callback to the globalworking_setthat ensures this method is called whenever a distribution is added to it.Therefore, you should not normally need to explicitly call this method.(Note that this means that namespace packages onsys.path are alwaysimported as soon aspkg_resources is, which is another reason whynamespace packages should not contain any code or import statements.)

as_requirement()

Return aRequirement instance that matches this distribution’s projectname and version.

requires(extras=())

List theRequirement objects that specify this distribution’sdependencies. Ifextras is specified, it should be a sequence of namesof “extras” defined by the distribution, and the list returned will theninclude any dependencies needed to support the named “extras”.

clone(**kw)

Create a copy of the distribution. Any supplied keyword arguments overridethe corresponding argument to theDistribution() constructor, allowingyou to change some of the copied distribution’s attributes.

egg_name()

Return what this distribution’s standard filename should be, not includingthe “.egg” extension. For example, a distribution for project “Foo”version 1.2 that runs on Python 2.3 for Windows would have anegg_name()ofFoo-1.2-py2.3-win32. Any dashes in the name or version areconverted to underscores. (Distribution.from_location() will convertthem back when parsing a “.egg” file name.)

__cmp__(other),__hash__()

Distribution objects are hashed and compared on the basis of their parsedversion and precedence, followed by their key (lowercase project name),location, Python version, and platform.

The following methods are used to accessEntryPoint objects advertisedby the distribution. See the section above onEntry Points for moredetailed information about these operations:

get_entry_info(group,name)

Return theEntryPoint object forgroup andname, or None if nosuch point is advertised by this distribution.

get_entry_map(group=None)

Return the entry point map forgroup. Ifgroup is None, returna dictionary mapping group names to entry point maps for all groups.(An entry point map is a dictionary of entry point names toEntryPointobjects.)

load_entry_point(group,name)

Short forget_entry_info(group,name).load(). Returns the objectadvertised by the named entry point, or raisesImportError ifthe entry point isn’t advertised by this distribution, or there is someother import problem.

In addition to the above methods,Distribution objects also implement allof theIResourceProvider andIMetadataProvider Methods (which aredocumented in later sections):

• has_metadata(name)

• metadata_isdir(name)

• metadata_listdir(name)

• get_metadata(name)

• get_metadata_lines(name)

• run_script(script_name,namespace)

• get_resource_filename(manager,resource_name)

• get_resource_stream(manager,resource_name)

• get_resource_string(manager,resource_name)

• has_resource(resource_name)

• resource_isdir(resource_name)

• resource_listdir(resource_name)

If the distribution was created with ametadata argument, these resource andmetadata access methods are all delegated to thatmetadata provider.Otherwise, they are delegated to anEmptyProvider, so that the distributionwill appear to have no resources or metadata. This delegation approach is usedso that supporting custom importers or new distribution formats can be donesimply by creating an appropriateIResourceProvider implementation; see thesection below onSupporting Custom Importers for more details.

Basic Resource Access¶

In the following methods, thepackage_or_requirement argument may be eithera Python package/module name (e.g.foo.bar) or aRequirement instance.If it is a package or module name, the named module or package must beimportable (i.e., be in a distribution or directory onsys.path), and theresource_name argument is interpreted relative to the named package. (Notethat if a module name is used, then the resource name is relative to thepackage immediately containing the named module. Also, you should not use usea namespace package name, because a namespace package can be spread acrossmultiple distributions, and is therefore ambiguous as to which distributionshould be searched for the resource.)

If it is aRequirement, then the requirement is automatically resolved(searching the currentEnvironment if necessary) and a matchingdistribution is added to theWorkingSet andsys.path if one was notalready present. (Unless theRequirement can’t be satisfied, in whichcase an exception is raised.) Theresource_name argument is then interpretedrelative to the root of the identified distribution; i.e. its first pathsegment will be treated as a peer of the top-level modules or packages in thedistribution.

Note that resource names must be/-separated paths rooted at the package,cannot contain relative names like"..", and cannot be absolute. Donot useos.path routines to manipulate resource paths, as they arenot filesystempaths.

resource_exists(package_or_requirement,resource_name)

Does the named resource exist? ReturnTrue orFalse accordingly.

resource_stream(package_or_requirement,resource_name)

Return a readable file-like object for the specified resource; it may bean actual file, aStringIO, or some similar object. The stream isin “binary mode”, in the sense that whatever bytes are in the resourcewill be read as-is.

resource_string(package_or_requirement,resource_name)

Return the specified resource asbytes. The resource is read inbinary fashion, such that the returned string contains exactly the bytesthat are stored in the resource.

resource_isdir(package_or_requirement,resource_name)

Is the named resource a directory? ReturnTrue orFalseaccordingly.

resource_listdir(package_or_requirement,resource_name)

List the contents of the named resource directory, just likeos.listdirexcept that it works even if the resource is in a zipfile.

Note that onlyresource_exists() andresource_isdir() are insensitiveas to the resource type. You cannot useresource_listdir() on a fileresource, and you can’t useresource_string() orresource_stream() ondirectory resources. Using an inappropriate method for the resource type mayresult in an exception or undefined behavior, depending on the platform anddistribution format involved.

Resource Extraction¶

resource_filename(package_or_requirement,resource_name)

Sometimes, it is not sufficient to access a resource in string or streamform, and a true filesystem filename is needed. In such cases, you canuse this method (or module-level function) to obtain a filename for aresource. If the resource is in an archive distribution (such as a zippedegg), it will be extracted to a cache directory, and the filename withinthe cache will be returned. If the named resource is a directory, thenall resources within that directory (including subdirectories) are alsoextracted. If the named resource is a C extension or “eager resource”(see thesetuptools documentation for details), then all C extensionsand eager resources are extracted at the same time.

Archived resources are extracted to a cache location that can be managed bythe following two methods:

set_extraction_path(path)

Set the base path where resources will be extracted to, if needed.

If you do not call this routine before any extractions take place, thepath defaults to the return value ofget_default_cache(). (Which isbased on thePYTHON_EGG_CACHE environment variable, with variousplatform-specific fallbacks. See that routine’s documentation for moredetails.)

Resources are extracted to subdirectories of this path based uponinformation given by the resource provider. You may set this to atemporary directory, but then you must callcleanup_resources() todelete the extracted files when done. There is no guarantee thatcleanup_resources() will be able to remove all extracted files. (OnWindows, for example, you can’t unlink .pyd or .dll files that are stillin use.)

Note that you may not change the extraction path for a given resourcemanager once resources have been extracted, unless you first callcleanup_resources().

cleanup_resources(force=False)

Delete all extracted resource files and directories, returning a listof the file and directory names that could not be successfully removed.This function does not have any concurrency protection, so it shouldgenerally only be called when the extraction path is a temporarydirectory exclusive to a single process. This method is notautomatically called; you must call it explicitly or register it as anatexit function if you wish to ensure cleanup of a temporarydirectory used for extractions.

“Provider” Interface¶

If you are implementing anIResourceProvider and/orIMetadataProviderfor a new distribution archive format, you may need to use the followingIResourceManager methods to coordinate extraction of resources to thefilesystem. If you’re not implementing an archive format, however, you haveno need to use these methods. Unlike the other methods listed above, they arenot available as top-level functions tied to the globalResourceManager;you must therefore have an explicitResourceManager instance to use them.

get_cache_path(archive_name,names=())

Return absolute location in cache forarchive_name andnames

The parent directory of the resulting path will be created if it doesnot already exist.archive_name should be the base filename of theenclosing egg (which may not be the name of the enclosing zipfile!),including its “.egg” extension.names, if provided, should be asequence of path name parts “under” the egg’s extraction location.

This method should only be called by resource providers that need toobtain an extraction location, and only for names they intend toextract, as it tracks the generated names for possible cleanup later.

extraction_error()

Raise anExtractionError describing the active exception as interferingwith the extraction process. You should call this if you encounter anyOS errors extracting the file to the cache path; it will format theoperating system exception for you, and add other information to theExtractionError instance that may be needed by programs that want towrap or handle extraction errors themselves.

postprocess(tempname,filename)

Perform any platform-specific postprocessing oftempname.Resource providers should call this method ONLY after successfullyextracting a compressed resource. They must NOT call it on resourcesthat are already in the filesystem.

tempname is the current (temporary) name of the file, andfilenameis the name it will be renamed to by the caller after this routinereturns.

IMetadataProvider Methods¶

The methods provided by objects (such asDistribution instances) thatimplement theIMetadataProvider orIResourceProvider interfaces are:

has_metadata(name)

Does the named metadata resource exist?

metadata_isdir(name)

Is the named metadata resource a directory?

metadata_listdir(name)

List of metadata names in the directory (likeos.listdir())

get_metadata(name)

Return the named metadata resource as a string. The data is read in binarymode; i.e., the exact bytes of the resource file are returned.

get_metadata_lines(name)

Yield named metadata resource as list of non-blank non-comment lines. Thisis short for callingyield_lines(provider.get_metadata(name)). See thesection onyield_lines() below for more information on the syntax itrecognizes.

run_script(script_name,namespace)

Execute the named script in the supplied namespace dictionary. RaisesResolutionError if there is no script by that name in thescriptsmetadata directory.namespace should be a Python dictionary, usuallya module dictionary if the script is being run as a module.

IResourceProvider¶

IResourceProvider is an abstract class that documents what methods arerequired of objects returned by aprovider_factory registered withregister_loader_type().IResourceProvider is a subclass ofIMetadataProvider, so objects that implement this interface must alsoimplement all of theIMetadataProvider Methods as well as the methodsshown here. Themanager argument to the methods below must be an objectthat supports the fullResourceManager API documented above.

get_resource_filename(manager,resource_name)

Return a true filesystem path forresource_name, coordinating theextraction withmanager, if the resource must be unpacked to thefilesystem.

get_resource_stream(manager,resource_name)

Return a readable file-like object forresource_name.

get_resource_string(manager,resource_name)

Return a string containing the contents ofresource_name.

has_resource(resource_name)

Does the package contain the named resource?

resource_isdir(resource_name)

Is the named resource a directory? Return a false value if the resourcedoes not exist or is not a directory.

resource_listdir(resource_name)

Return a list of the contents of the resource directory, alaos.listdir(). Requesting the contents of a non-existent directory mayraise an exception.

Note, by the way, that your provider classes need not (and should not) subclassIResourceProvider orIMetadataProvider! These classes exist solelyfor documentation purposes and do not provide any useful implementation code.You may instead wish to subclass one of thebuilt-in resource providers.

Built-in Resource Providers¶

pkg_resources includes several provider classes that are automatically usedwhere appropriate. Their inheritance tree looks like this:

NullProviderEggProviderDefaultProviderPathMetadataZipProviderEggMetadataEmptyProviderFileMetadata

NullProvider

This provider class is just an abstract base that provides for commonprovider behaviors (such as running scripts), given a definition for justa few abstract methods.

EggProvider

This provider class adds in some egg-specific features that are commonto zipped and unzipped eggs.

DefaultProvider

This provider class is used for unpacked eggs and “plain old Python”filesystem modules.

ZipProvider

This provider class is used for all zipped modules, whether they are eggsor not.

EmptyProvider

This provider class always returns answers consistent with a provider thathas no metadata or resources.Distribution objects created withoutametadata argument use an instance of this provider class instead.Since allEmptyProvider instances are equivalent, there is no needto have more than one instance.pkg_resources therefore creates aglobal instance of this class under the nameempty_provider, and youmay use it if you have need of anEmptyProvider instance.

PathMetadata(path,egg_info)

Create anIResourceProvider for a filesystem-based distribution, wherepath is the filesystem location of the importable modules, andegg_infois the filesystem location of the distribution’s metadata directory.egg_info should usually be theEGG-INFO subdirectory ofpath for an“unpacked egg”, and aProjectName.egg-info subdirectory ofpath fora “development egg”. However, other uses are possible for custom purposes.

EggMetadata(zipimporter)

Create anIResourceProvider for a zipfile-based distribution. Thezipimporter should be azipimport.zipimporter instance, and mayrepresent a “basket” (a zipfile containing multiple “.egg” subdirectories)a specific eggwithin a basket, or a zipfile egg (where the zipfileitself is a “.egg”). It can also be a combination, such as a zipfile eggthat also contains other eggs.

FileMetadata(path_to_pkg_info)

Create anIResourceProvider that provides exactly one metadataresource:PKG-INFO. The supplied path should be a distutils PKG-INFOfile. This is basically the same as anEmptyProvider, except thatrequests forPKG-INFO will be answered using the contents of thedesignated file. (This provider is used to wrap.egg-info filesinstalled by vendor-supplied system packages.)

Parsing Utilities¶

parse_version(version)

Parsed a project’s version string as defined by PEP 440. The returnedvalue will be an object that represents the version. These objects maybe compared to each other and sorted. The sorting algorithm is as definedby PEP 440 with the addition that any version which is not a valid PEP 440version will be considered less than any valid PEP 440 version and theinvalid versions will continue sorting using the original algorithm.

yield_lines(strs)

Yield non-empty/non-comment lines from a string/unicode or apossibly-nested sequence thereof. Ifstrs is an instance ofbasestring, it is split into lines, and each non-blank, non-commentline is yielded after stripping leading and trailing whitespace. (Lineswhose first non-blank character is# are considered comment lines.)

Ifstrs is not an instance ofbasestring, it is iterated over, andeach item is passed recursively toyield_lines(), so that an arbitrarilynested sequence of strings, or sequences of sequences of strings can beflattened out to the lines contained therein. So for example, passinga file object or a list of strings toyield_lines will both work.(Note that between each string in a sequence of strings there is assumed tobe an implicit line break, so lines cannot bridge two strings in asequence.)

This routine is used extensively bypkg_resources to parse metadataand file formats of various kinds, and most otherpkg_resourcesparsing functions that yield multiple values will use it to break up theirinput. However, this routine is idempotent, so callingyield_lines()on the output of another call toyield_lines() is completely harmless.

split_sections(strs)

Split a string (or possibly-nested iterable thereof), yielding(section,content) pairs found using an.ini-like syntax. Eachsection isa whitespace-stripped version of the section name (”[section]”)and eachcontent is a list of stripped lines excluding blank lines andcomment-only lines. If there are any non-blank, non-comment lines beforethe first section header, they’re yielded in a firstsection ofNone.

This routine usesyield_lines() as its front end, so you can pass inanything thatyield_lines() accepts, such as an open text file, string,or sequence of strings.ValueError is raised if a malformed sectionheader is found (i.e. a line starting with[ but not ending with]).

Note that this simplistic parser assumes that any line whose first nonblankcharacter is[ is a section heading, so it can’t support .ini formatvariations that allow[ as the first nonblank character on other lines.

safe_name(name)

Return a “safe” form of a project’s name, suitable for use in aRequirement string, as a distribution name, or a PyPI project name.All non-alphanumeric runs are condensed to single “-” characters, such thata name like “The $$$ Tree” becomes “The-Tree”. Note that if you aregenerating a filename from this value you should combine it with a call toto_filename() so all dashes (“-”) are replaced by underscores (“_”).Seeto_filename().

safe_version(version)

This will return the normalized form of any PEP 440 version. If the versionstring is not PEP 440 compatible, this function behaves similar tosafe_name() except that spaces in the input become dots, and dots areallowed to exist in the output. As withsafe_name(), if you aregenerating a filename from this you should replace any “-” characters inthe output with underscores.

safe_extra(extra)

Return a “safe” form of an extra’s name, suitable for use in a requirementstring or a setup script’sextras_require keyword. This routine issimilar tosafe_name() except that non-alphanumeric runs are replacedby a single underbar (_), and the result is lowercased.

to_filename(name_or_version)

Escape a name or version string so it can be used in a dash-separatedfilename (or#egg=name-version tag) without ambiguity. Youshould only pass in values that were returned bysafe_name() orsafe_version().

Platform Utilities¶

get_build_platform()

Return this platform’s identifier string. For Windows, the return valueis"win32", and for macOS it is a string of the form"macosx-10.4-ppc". All other platforms return the same uname-basedstring that thedistutils.util.get_platform() function returns.This string is the minimum platform version required by distributions builton the local machine. (Backward compatibility note: setuptools versionsprior to 0.6b1 called this functionget_platform(), and the function isstill available under that name for backward compatibility reasons.)

get_supported_platform() (New in 0.6b1)

This is the similar toget_build_platform(), but is the maximumplatform version that the local machine supports. You will usually wantto use this value as theprovided argument to thecompatible_platforms() function.

compatible_platforms(provided,required)

Return true if a distribution built on theprovided platform may be usedon therequired platform. If either platform value isNone, it isconsidered a wildcard, and the platforms are therefore compatible.Likewise, if the platform strings are equal, they’re also consideredcompatible, andTrue is returned. Currently, the only non-equalplatform strings that are considered compatible are macOS platformstrings with the same hardware type (e.g.ppc) and major version(e.g.10) with theprovided platform’s minor version being less thanor equal to therequired platform’s minor version.

get_default_cache()

Determine the default cache location for extracting resources from zippedeggs. This routine returns thePYTHON_EGG_CACHE environment variable,if set. Otherwise, on Windows, it returns a “Python-Eggs” subdirectory ofthe user’s “Application Data” directory. On all other systems, it returnsos.path.expanduser("~/.python-eggs") ifPYTHON_EGG_CACHE is notset.

PEP 302 Utilities¶

get_importer(path_item)

A deprecated alias forpkgutil.get_importer()

File/Path Utilities¶

ensure_directory(path)

Ensure that the parent directory (os.path.dirname) ofpath actuallyexists, usingos.makedirs() if necessary.

normalize_path(path)

Return a “normalized” version ofpath, such that two paths representthe same filesystem location if they have equalnormalized_path()values. Specifically, this is a shortcut for callingos.path.realpathandos.path.normcase onpath. Unfortunately, on certain platforms(notably Cygwin and macOS) thenormcase function does not accuratelyreflect the platform’s case-sensitivity, so there is always the possibilityof two apparently-different paths being equal on such platforms.

History¶

0.6c9

• Fixresource_listdir('') always returning an empty list for zipped eggs.

0.6c7

• Fix package precedence problem where single-version eggs installed insite-packages would take precedence over.egg files (or directories)installed insite-packages.

0.6c6

• Fix extracted C extensions not having executable permissions under Cygwin.

• Allow.egg-link files to contain relative paths.

• Fix cache dir defaults on Windows when multiple environment vars are neededto construct a path.

0.6c4

• Fix “dev” versions being considered newer than release candidates.

0.6c3

• Python 2.5 compatibility fixes.

0.6c2

• Fix a problem with eggs specified directly onPYTHONPATH oncase-insensitive filesystems possibly not showing up in the defaultworking set, due to differing normalizations ofsys.path entries.

0.6b3

• Fixed a duplicate path insertion problem on case-insensitive filesystems.

0.6b1

• Splitget_platform() intoget_supported_platform() andget_build_platform() to work around a Mac versioning problem that causedthe behavior ofcompatible_platforms() to be platform specific.

• Fix entry point parsing when a standalone module name has whitespacebetween it and the extras.

0.6a11

• AddedExtractionError andResourceManager.extraction_error() so thatcache permission problems get a more user-friendly explanation of theproblem, and so that programs can catch and handle extraction errors if theyneed to.

0.6a10

• Added theextras attribute toDistribution, thefind_plugins()method toWorkingSet, and the__add__() and__iadd__() methodstoEnvironment.

• safe_name() now allows dots in project names.

• There is a newto_filename() function that escapes project names andversions for safe use in constructing egg filenames from a Distributionobject’s metadata.

• AddedDistribution.clone() method, and keyword argument support to otherDistribution constructors.

• Added theDEVELOP_DIST precedence, and automatically assign it toeggs using.egg-info format.

0.6a9

• Don’t raise an error when an invalid (unfinished) distribution is foundunless absolutely necessary. Warn about skipping invalid/unfinished eggswhen building an Environment.

• Added support for.egg-info files or directories with version/platforminformation embedded in the filename, so that system packagers have theoption of includingPKG-INFO files to indicate the presence of asystem-installed egg, without needing to use.egg directories, zipfiles,or.pth manipulation.

• Changedparse_version() to remove dashes before pre-release tags, sothat0.2-rc1 is considered anolder version than0.2, and is equalto0.2rc1. The idea that a dashalways meant a post-release versionwas highly non-intuitive to setuptools users and Python developers, whoseem to want to use-rc version numbers a lot.

0.6a8

• Fixed a problem withWorkingSet.resolve() that prevented versionconflicts from being detected at runtime.

• Improved runtime conflict warning message to identify a line in the user’sprogram, rather than flagging thewarn() call inpkg_resources.

• Avoid giving runtime conflict warnings for namespace packages, even if theywere declared by a different package than the one currently being activated.

• Fix path insertion algorithm for case-insensitive filesystems.

• Fixed a problem with nested namespace packages (e.g.peak.util) notbeing set as an attribute of their parent package.

0.6a6

• Activated distributions are now inserted insys.path (and the workingset) just before the directory that contains them, instead of at the end.This allows e.g. eggs insite-packages to override unmanaged modules inthe same location, and allows eggs found earlier onsys.path to overrideones found later.

• When a distribution is activated, it now checks whether any containednon-namespace modules have already been imported and issues a warning ifa conflicting module has already been imported.

• Changed dependency processing so that it’s breadth-first, allowing adepender’s preferences to override those of a dependee, to prevent conflictswhen a lower version is acceptable to the dependee, but not the depender.

• Fixed a problem extracting zipped files on Windows, when the egg in questionhas had changed contents but still has the same version number.

0.6a4

• Fix a bug inWorkingSet.resolve() that was introduced in 0.6a3.

0.6a3

• Addedsafe_extra() parsing utility routine, and use it for Requirement,EntryPoint, and Distribution objects’ extras handling.

0.6a1

• Enhanced performance ofrequire() and related operations when allrequirements are already in the working set, and enhanced performance ofdirectory scanning for distributions.

• Fixed some problems usingpkg_resources w/PEP 302 loaders other thanzipimport, and the previously-broken “eager resource” support.

• Fixedpkg_resources.resource_exists() not working correctly, along withsome other resource API bugs.

• Many API changes and enhancements:

  • AddedEntryPoint,get_entry_map,load_entry_point, andget_entry_info APIs for dynamic plugin discovery.

  • list_resources is nowresource_listdir (and it actually works)

  • Resource API functions likeresource_string() that accepted a packagename and resource name, will now also accept aRequirement object inplace of the package name (to allow access to non-package data files inan egg).

  • get_provider() will now accept aRequirement instance or a modulename. If it is given aRequirement, it will return a correspondingDistribution (by callingrequire() if a suitable distributionisn’t already in the working set), rather than returning a metadata andresource provider for a specific module. (The difference is in howresource paths are interpreted; supplying a module name means resourcespath will be module-relative, rather than relative to the distribution’sroot.)

  • Distribution objects now implement theIResourceProvider andIMetadataProvider interfaces, so you don’t need to reference the (nolonger available)metadata attribute to get at these interfaces.

  • Distribution andRequirement both have aproject_nameattribute for the project name they refer to. (Previously these werename anddistname attributes.)

  • Thepath attribute ofDistribution objects is nowlocation,because it isn’t necessarily a filesystem path (and hasn’t been for sometime now). Thelocation ofDistribution objects in the filesystemshould always be normalized usingpkg_resources.normalize_path(); allof the setuptools’ code that generates distributions from the filesystem(includingDistribution.from_filename()) ensure this invariant, but ifyou use a more generic API likeDistribution() orDistribution.from_location() you should take care that you don’tcreate a distribution with an un-normalized filesystem path.

  • Distribution objects now have anas_requirement() method thatreturns aRequirement for the distribution’s project name and version.

  • Distribution objects no longer have aninstalled_on() method, and theinstall_on() method is nowactivate() (but may go away altogethersoon). Thedepends() method has also been renamed torequires(),andInvalidOption is nowUnknownExtra.

  • find_distributions() now takes an additional argument calledonly,that tells it to only yield distributions whose location is the passed-inpath. (It defaults to False, so that the default behavior is unchanged.)

  • AvailableDistributions is now calledEnvironment, and theget(),__len__(), and__contains__() methods were removed,because they weren’t particularly useful.__getitem__() no longerraisesKeyError; it just returns an empty list if there are nodistributions for the named project.

  • Theresolve() method ofEnvironment is now a method ofWorkingSet instead, and thebest_match() method now uses a workingset instead of a path list as its second argument.

  • There is a newpkg_resources.add_activation_listener() API that letsyou register a callback for notifications about distributions added tosys.path (including the distributions already on it). This isbasically a hook for extensible applications and frameworks to be able tosearch for plugin metadata in distributions added at runtime.

0.5a13

• Fixed a bug in resource extraction from nested packages in a zipped egg.

0.5a12

• Updated extraction/cache mechanism for zipped resources to avoidinter-process and inter-thread races during extraction. The default cachelocation can now be set via thePYTHON_EGGS_CACHE environment variable,and the default Windows cache is now aPython-Eggs subdirectory of thecurrent user’s “Application Data” directory, if thePYTHON_EGGS_CACHEvariable isn’t set.

0.5a10

• Fix a problem withpkg_resources being confused by non-existent eggs onsys.path (e.g. if a user deletes an egg without removing it from theeasy-install.pth file).

• Fix a problem with “basket” support inpkg_resources, where egg-findingnever actually went inside.egg files.

• Madepkg_resources import the module you request resources from, if it’snot already imported.

0.5a4

• pkg_resources.AvailableDistributions.resolve() and related methods nowaccept aninstaller argument: a callable taking one argument, aRequirement instance. The callable must return aDistributionobject, orNone if no distribution is found. This feature is used byEasyInstall to resolve dependencies by recursively invoking itself.

0.4a4

• Fix problems withresource_listdir(),resource_isdir() and resourcedirectory extraction for zipped eggs.

0.4a3

• Fixed scripts not being able to see a__file__ variable in__main__

• Fixed a problem withresource_isdir() implementation that was introducedin 0.4a2.

0.4a1

• Fixed a bug in requirements processing for exact versions (i.e.== and!=) when only one condition was included.

• Addedsafe_name() andsafe_version() APIs to clean up handling ofarbitrary distribution names and versions found on PyPI.

0.3a4

• pkg_resources now supports resource directories, not just the resourcesin them. In particular, there areresource_listdir() andresource_isdir() APIs.

• pkg_resources now supports “egg baskets” – .egg zipfiles which containmultiple distributions in subdirectories whose names end with.egg.Having such a “basket” in a directory onsys.path is equivalent tohaving the individual eggs in that directory, but the contained eggs canbe individually added (or not) tosys.path. Currently, however, thereis no automated way to create baskets.

• Namespace package manipulation is now protected by the Python import lock.

0.3a1

• Initial release.