This PR adds theLicense-File field to the package metadata. Although not yet part of the official metadata spec, including this field will enable third party tools easier access to all included license files throughimportlib.metadata. This is especially useful if a file doesn't have an easily recognizable name or is in an unusual location.

setuptools already parses thelicense_file andlicense_files options and cross-references them with all project files to see which ones should be included. With this change, the result of that would be made available for others to use as well.

Closes#2631

--
For reference:PEP 639, currently still in a draft, plans to add theLicense-File field to the metadata spec,link

--
Example use case

fromimportlibimportmetadatameta=metadata.metadata('my-project')project_files=metadata.files('my-project')license_file_paths= [pathforpathinproject_filesifstr(path)inmeta.get_all('License-File')]

Open questions

1. What should the value ofLicense-Path be for files not in main project directory?
From a technical standpoint it would probably be best to use the actual relative path and mirror that in the.dist-info directory. However, at the momentwheel flattens the folder structure and just puts all license files into.dist-info, AFAIK. This could lead to name conflicts.

2. Should license files be copied into the.egg-info directory if the legacy install is used?

3. How shouldlicense_files andMANIFEST.in continue to work together?
At the moment, theMANIFEST.in is parse at the end and thus overwrites all previous file includes. However,license_files_computed will only look at thelicense_file andlicense_files options (unless we parse the manifest there too). That means that theLicense-File field might contain files that are later excluded by the manifest.
-> I would recommend to change the behavior so that thelicense_file andlicense_files options take precedent over the manifest. A license file is most likely excluded by accident anyway. Additionally, specifyinglicense_files = (without any values) won't add any default patterns and thus not match any license files.

Pull Request Checklist

Changes have tests
News fragment added inchangelog.d/.
(Seedocumentation for details)

cdce8p force-pushed thelicense-file branch froma09ff3c to20407bfCompare

April 17, 2021 19:23

cdce8p force-pushed thelicense-file branch from20407bf to8d2c7beCompare

May 9, 2021 18:32

jaraco reviewed

May 19, 2021

View reviewed changes

setuptools/command/sdist.pyShow resolvedHide resolved

jaraco reviewed

May 19, 2021

View reviewed changes

setuptools/dist.py OutdatedShow resolvedHide resolved

jaraco reviewed

May 19, 2021

View reviewed changes

setuptools/dist.py Outdated

		ifpathnotinfilesandos.path.isfile(path):
		files.add(path)

		self.metadata.license_files_computed=sorted(files)

Copy link

Member

jaracoMay 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Can we retain order of license files rather than enforcing order? For the manifest, that's less important, but for metadata, it may prove useful to honor the user's supplied order.

Copy link

MemberAuthor

cdce8pMay 20, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

The user only provides patterns which are used forglob. As such, I don't think retaining any order would make much sense.

Copy link

Member

jaracoMay 21, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Alright. I'll probably re-write it to avoid sorting or at least avoid sorting user-supplied values.

jaraco approved these changes

May 19, 2021

View reviewed changes

Copy link

Member

jaraco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I like where this is going. Just a couple of concerns.

Copy link

MemberAuthor

cdce8p commentedMay 20, 2021

I've updated the PR. Additionally, I also added a commit to change when license files are added to the source list. As explained in the open questions section (3. How should license_files and MANIFEST.in continue to work together?), to make sure a license file which is listed inLicense-File is included, thelicense_files option needs to be able to overwrite theMANIFEST.in

jaraco reviewed

May 21, 2021

View reviewed changes

setuptools/dist.py Outdated

		'project_urls':dict,
		'provides_extras':ordered_set.OrderedSet,
		'license_files':ordered_set.OrderedSet,
		'license_file':None,

Copy link

Member

jaracoMay 21, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I had kind-of hoped this would go away, especially since the value is deprecated. I tried removing it and got an error, but I think it can be eliminated by being more defensive at the one point where it's queried.

Copy link

MemberAuthor

cdce8pMay 21, 2021•
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I'm happy to remove it if you want. Personally, I would also be open to removing the optionlicense_file entirely and return an error. Should be easy enough for the user to fix then.

jaraco reviewed

May 21, 2021

View reviewed changes

setuptools/dist.py Outdated

		'provides_extras':ordered_set.OrderedSet,
		'license_files':ordered_set.OrderedSet,
		'license_file':None,
		'license_files':None,

Copy link

Member

jaracoMay 21, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Here it probably makes sense to just declare the default here rather than declare a null value and then set a default if the value is a null value.

Copy link

MemberAuthor

cdce8pMay 21, 2021•
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

That was my initial idea as well. However, with that we can't determined if the user didn't specify the option (thus the defaults would be loaded) or left it empty (in which case no license files would be included). Both would equal an empty list.

Copy link

Member

jaracoMay 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Oh, that's very interesting. And the test cases capture this aspect. Consider me impressed.

Copy link

Member

jaraco commentedMay 21, 2021

I added a couple more comments, but I'll take the first stab at adjusting the behavior.

cdce8p force-pushed thelicense-file branch 2 times, most recently from91f8e31 toaf7b7b5Compare

May 22, 2021 10:48

Copy link

MemberAuthor

cdce8p commentedMay 22, 2021

@jaraco Pushed new change to address some of your points. First, I modified the logic to match license files so that the user sorting is kept as best as possible. Second, I removed the defaultlicense_file metadata attribute. Let me know if you what me to change anything else.

Copy link

MemberAuthor

cdce8p commentedMay 22, 2021

Small update: I needed to revert the removal of thelicense_file attribute. Without it the deprecation warning wouldn't be printed (even with-Wd).

Copy link

Member

jaraco commentedMay 22, 2021

I spent some time trying to simplify the logic by having the defaults for "license_files" be defined in the defaults for license files (rather than having a null value as the default and defining defaults in the finalizer), but when doing so, I encountered two problems:

The config parserwon't override a default with an explicit value ifbool(default) == True.
The logic for the default/fallback values is contingent both onlicense_file andlicense_files not being specified, so untillicense_file can be removed, the two settings are inherently entangled.

At some point soon, I'd like to remove support for license_file and remove that check for overriding a non-False default. For now this should do.

I've added one additional commit that removes the sorting entirely and uses a generator to build the list. I see I don't have push access to the PR, so I'll try sending a PR to your branch. If that doesn't work, I'll probably just merge it all together manually.

jaraco mentioned this pull request

May 22, 2021

Replace for/if/add/extend with generator on patterns.cdce8p/setuptools#1

Merged

cdce8pand others added8 commits

May 23, 2021 01:42

Add License-File field to package metadata

3544de7

Remove license_files_computed field

b16725a

Overwrite exlude from MANIFEST with license_files option

482e8e7

* needed for 'License-File' metadata, as this is written  before MANIFEST is read

Keep user sorting for license files

3d1cce5

Fix after rebase

e8d7f88

Remove license_file

540a912

Revert removal of license_file

c837956

Replace for/if/add/extend with generator on patterns. Use unique_ever…

4429ffb

…seen to dedupe.

cdce8p force-pushed thelicense-file branch from41cc2eb to4429ffbCompare

May 22, 2021 23:42

Fix flaky test

acf4807

Copy link

MemberAuthor

cdce8p commentedMay 22, 2021•
edited
Loading

I've added one additional commit that removes the sorting entirely and uses a generator to build the list. I see I don't have push access to the PR, so I'll try sending a PR to your branch. If that doesn't work, I'll probably just merge it all together manually.

Strange. I've allowed edits / commits by maintainers 🤔 Anyway, I merged your changes and, while I was add it, rebased the PR to resolve the merge conflict.

jaraco merged commit75e6496 intopypa:main

May 23, 2021

cdce8p deleted the license-file branch

May 23, 2021 00:11

cdce8p mentioned this pull request

May 23, 2021

(Frozen release plans) Progress to 4.0raimon49/pip-licenses#89

Closed

13 tasks

Copy link

Member

jaraco commentedMay 23, 2021

Strange. I've allowed edits / commits by maintainers 🤔 Anyway, I merged your changes and, while I was add it, rebased the PR to resolve the merge conflict.

Seems the mistake is on my side, an artifact of having switched fromhub togh andgh not handling pr checkouts as elegantly (cli/cli#2189).