How to make a good bug report#

When you submit an issue toGitHub, please do your best tofollow these guidelines! This will make it a lot easier to provide you with goodfeedback:

• The ideal bug report contains ashort reproducible code snippet, this way anyone can try to reproduce the bug easily. If yoursnippet is longer than around 50 lines, please link to aGist or a GitHub repo.

• If not feasible to include a reproducible snippet, please be specific aboutwhatestimators and/or functions are involved and the shape of the data.

• If an exception is raised, pleaseprovide the full traceback.

• Please include youroperating system type and version number, as well asyourPython, scikit-learn, numpy, and scipy versions. This informationcan be found by running:

  python-c"import sklearn; sklearn.show_versions()"

• Please ensure allcode snippets and error messages are formatted inappropriate code blocks. SeeCreating and highlighting code blocksfor more details.

If you want to help curate issues, read aboutBug triaging and issue curation.

Video resources#

These videos are step-by-step introductions on how to contribute toscikit-learn, and are a great companion to the following text guidelines.Please make sure to still check our guidelines below, since they describe ourlatest up-to-date workflow.

• Crash Course in Contributing to Scikit-Learn & Open Source Projects:Video,Transcript

• Example of Submitting a Pull Request to scikit-learn:Video,Transcript

• Sprint-specific instructions and practical tips:Video,Transcript

• 3 Components of Reviewing a Pull Request:Video,Transcript

How to contribute#

The preferred way to contribute to scikit-learn is to fork themainrepository on GitHub,then submit a “pull request” (PR).

In the first few steps, we explain how to locally install scikit-learn, andhow to set up your git repository:

1. Create an account onGitHub if you do not already have one.

2. Fork theproject repository: click on the ‘Fork’button near the top of the page. This creates a copy of the code under youraccount on the GitHub user account. For more details on how to fork arepository seethis guide.

3. Clone your fork of the scikit-learn repo from your GitHub account to yourlocal disk:

   gitclonegit@github.com:YourLogin/scikit-learn.git# add --depth 1 if your connection is slowcdscikit-learn

4. Follow steps 2-6 inBuilding from source to build scikit-learn indevelopment mode and return to this document.

5. Install the development dependencies:

   pipinstallpytestpytest-covruff==0.11.2mypynumpydoc

6. Add theupstream remote. This saves a reference to the mainscikit-learn repository, which you can use to keep your repositorysynchronized with the latest changes:

   gitremoteaddupstreamgit@github.com:scikit-learn/scikit-learn.git

7. Check that theupstream andorigin remote aliases are configured correctlyby running:

   gitremote-v

   This should display:

   origin    git@github.com:YourLogin/scikit-learn.git (fetch)origin    git@github.com:YourLogin/scikit-learn.git (push)upstream  git@github.com:scikit-learn/scikit-learn.git (fetch)upstream  git@github.com:scikit-learn/scikit-learn.git (push)

You should now have a working installation of scikit-learn, and your git repositoryproperly configured. It could be useful to run some test to verify your installation.Please refer toUseful pytest aliases and flags for examples.

The next steps now describe the process of modifying code and submitting a PR:

8. Synchronize yourmain branch with theupstream/main branch,more details onGitHub Docs:

   gitcheckoutmaingitfetchupstreamgitmergeupstream/main

9. Create a feature branch to hold your development changes:

   gitcheckout-bmy_feature

   and start making changes. Always use a feature branch. It’s goodpractice to never work on themain branch!

10. (Optional) Installpre-commit torun code style checks before each commit:

    pipinstallpre-commitpre-commitinstall

    pre-commit checks can be disabled for a particular commit withgitcommit-n.

11. Develop the feature on your feature branch on your computer, using Git todo the version control. When you’re done editing, add changed files usinggitadd and thengitcommit:

    gitaddmodified_filesgitcommit

    to record your changes in Git, then push the changes to your GitHubaccount with:

    gitpush-uoriginmy_feature

12. Followtheseinstructions to create a pull request from your fork. This will send anotification to potential reviewers. You may want to consider sending a message tothediscord in the developmentchannel for more visibility if your pull request does not receive attention aftera couple of days (instant replies are not guaranteed though).

It is often helpful to keep your local feature branch synchronized with thelatest changes of the main scikit-learn repository:

gitfetchupstreamgitmergeupstream/main

Subsequently, you might need to solve the conflicts. You can refer to theGit documentation related to resolving merge conflict using the commandline.

TheGit documentation andhttp://try.github.io are excellent resources to get started with git,and understanding all of the commands shown here.

Pull request checklist#

Before a PR can be merged, it needs to be approved by two core developers.An incomplete contribution – where you expect to do more work before receivinga full review – should be marked as adraft pull requestand changed to “ready for review” when it matures. Draft PRs may be useful to:indicate you are working on something to avoid duplicated work, requestbroad review of functionality or API, or seek collaborators. Draft PRs oftenbenefit from the inclusion of atask list inthe PR description.

In order to ease the reviewing process, we recommend that your contributioncomplies with the following rules before marking a PR as “ready for review”. Thebolded ones are especially important:

1. Give your pull request a helpful title that summarizes what yourcontribution does. This title will often become the commit message oncemerged so it should summarize your contribution for posterity. In somecases “Fix <ISSUE TITLE>” is enough. “Fix #<ISSUE NUMBER>” is never agood title.

2. Make sure your code passes the tests. The whole test suite can be runwithpytest, but it is usually not recommended since it takes a longtime. It is often enough to only run the test related to your changes:for example, if you changed something insklearn/linear_model/_logistic.py, running the following commands willusually be enough:

   • pytestsklearn/linear_model/_logistic.py to make sure the doctestexamples are correct

   • pytestsklearn/linear_model/tests/test_logistic.py to run the testsspecific to the file

   • pytestsklearn/linear_model to test the wholelinear_model module

   • pytestdoc/modules/linear_model.rst to make sure the user guideexamples are correct.

   • pytestsklearn/tests/test_common.py-kLogisticRegression to run all ourestimator checks (specifically forLogisticRegression, if that’s theestimator you changed).

   There may be other failing tests, but they will be caught by the CI soyou don’t need to run the whole test suite locally. For guidelines on howto usepytest efficiently, see theUseful pytest aliases and flags.

3. Make sure your code is properly commented and documented, andmakesure the documentation renders properly. To build the documentation, pleaserefer to ourDocumentation guidelines. The CI will alsobuild the docs: please refer toGenerated documentation on GitHub Actions.

4. Tests are necessary for enhancements to beaccepted. Bug-fixes or new features should be provided withnon-regression tests. These testsverify the correct behavior of the fix or feature. In this manner, furthermodifications on the code base are granted to be consistent with thedesired behavior. In the case of bug fixes, at the time of the PR, thenon-regression tests should fail for the code base in themain branchand pass for the PR code.

5. If your PR is likely to affect users, you need to add a changelog entry describingyour PR changes. See theREADMEfor more details.

6. Follow theCoding guidelines.

7. When applicable, use the validation tools and scripts in thesklearn.utilsmodule. A list of utility routines available for developers can be found in theUtilities for Developers page.

8. Often pull requests resolve one or more other issues (or pull requests).If merging your pull request means that some other issues/PRs shouldbe closed, you shoulduse keywords to create link to them(e.g.,Fixes#1234; multiple issues/PRs are allowed as long as eachone is preceded by a keyword). Upon merging, those issues/PRs willautomatically be closed by GitHub. If your pull request is simplyrelated to some other issues/PRs, or it only partially resolves the targetissue, create a link to them without using the keywords (e.g.,Towards#1234).

9. PRs should often substantiate the change, through benchmarks ofperformance and efficiency (seeMonitoring performance) or throughexamples of usage. Examples also illustrate the features and intricacies ofthe library to users. Have a look at other examples in theexamples/directory for reference. Examples should demonstrate why the newfunctionality is useful in practice and, if possible, compare it to othermethods available in scikit-learn.

10. New features have some maintenance overhead. We expect PR authorsto take part in the maintenance for the code they submit, at leastinitially. New features need to be illustrated with narrativedocumentation in the user guide, with small code snippets.If relevant, please also add references in the literature, with PDF linkswhen possible.

11. The user guide should also include expected time and space complexityof the algorithm and scalability, e.g. “this algorithm can scale to alarge number of samples > 100000, but does not scale in dimensionality:n_features is expected to be lower than 100”.

You can also check ourCode Review Guidelines to get an idea of what reviewerswill expect.

You can check for common programming errors with the following tools:

• Code with a good unit test coverage (at least 80%, better 100%), check with:

  pipinstallpytestpytest-covpytest--covsklearnpath/to/tests

  See alsoTesting and improving test coverage.

• Run static analysis withmypy:

  mypysklearn

  This must not produce new errors in your pull request. Using#type:ignoreannotation can be a workaround for a few cases that are not supported bymypy, in particular,

  • when importing C or Cython modules,

  • on properties with decorators.

Bonus points for contributions that include a performance analysis witha benchmark script and profiling output (seeMonitoring performance).Also check out theHow to optimize for speed guide for more details onprofiling and Cython optimizations.

Continuous Integration (CI)#

• Azure pipelines are used for testing scikit-learn on Linux, Mac and Windows,with different dependencies and settings.

• CircleCI is used to build the docs for viewing.

• Github Actions are used for various tasks, including building wheels andsource distributions.

# pull latest upstream/maingitpullupstreammain--no-rebase# resolve conflicts - keeping the upstream/main version for specific filesgitcheckout--theirsbuild_tools/*/*.lockbuild_tools/*/*environment.yml\build_tools/*/*lock.txtbuild_tools/*/*requirements.txtgitaddbuild_tools/*/*.lockbuild_tools/*/*environment.yml\build_tools/*/*lock.txtbuild_tools/*/*requirements.txtgitmerge--continue

pythonbuild_tools/update_environments_and_lock_files.py

Stalled pull requests#

As contributing a feature can be a lengthy process, somepull requests appear inactive but unfinished. In such a case, takingthem over is a great service for the project. A good etiquette to take over is:

• Determine if a PR is stalled

  • A pull request may have the label “stalled” or “help wanted” if wehave already identified it as a candidate for other contributors.

  • To decide whether an inactive PR is stalled, ask the contributor ifshe/he plans to continue working on the PR in the near future.Failure to respond within 2 weeks with an activity that moves the PRforward suggests that the PR is stalled and will result in taggingthat PR with “help wanted”.

    Note that if a PR has received earlier comments on the contributionthat have had no reply in a month, it is safe to assume that the PRis stalled and to shorten the wait time to one day.

    After a sprint, follow-up for un-merged PRs opened during sprint willbe communicated to participants at the sprint, and those PRs will betagged “sprint”. PRs tagged with “sprint” can be reassigned ordeclared stalled by sprint leaders.

• Taking over a stalled PR: To take over a PR, it is important tocomment on the stalled PR that you are taking over and to link from thenew PR to the old one. The new PR should be created by pulling from theold one.

Stalled and Unclaimed Issues#

Generally speaking, issues which are up for grabs will have a“help wanted”.tag. However, not all issues which need contributors will have this tag,as the “help wanted” tag is not always up-to-date with the stateof the issue. Contributors can find issues which are still up for grabsusing the following guidelines:

• First, todetermine if an issue is claimed:

  • Check for linked pull requests

  • Check the conversation to see if anyone has said that they’re working oncreating a pull request

• If a contributor comments on an issue to say they are working on it,a pull request is expected within 2 weeks (new contributor) or 4 weeks(contributor or core dev), unless a larger time frame is explicitly given.Beyond that time, another contributor can take the issue and make apull request for it. We encourage contributors to comment directly on thestalled or unclaimed issue to let community members know that they will beworking on it.

• If the issue is linked to astalled pull request,we recommend that contributors follow the proceduredescribed in theStalled pull requestssection rather than working directly on the issue.

Issues for New Contributors#

New contributors should look for the following tags when looking for issues. Westrongly recommend that new contributors tackle “easy” issues first: this helpsthe contributor become familiar with the contribution workflow, and for the coredevs to become acquainted with the contributor; besides which, we frequentlyunderestimate how easy an issue is to solve!

• Good first issue tag

  A great way to start contributing to scikit-learn is to pick an item fromthe list ofgood first issuesin the issue tracker. Resolving these issues allows you to start contributingto the project without much prior knowledge. If you have already contributedto scikit-learn, you should look at Easy issues instead.

• Easy tag

  If you have already contributed to scikit-learn, another great way to contributeto scikit-learn is to pick an item from the list ofEasy issues in the issuetracker. Your assistance in this area will be greatly appreciated by themore experienced developers as it helps free up their time to concentrate onother issues.

• Help wanted tag

  We often use the help wanted tag to mark issues regardless of difficulty.Additionally, we use the help wanted tag to mark Pull Requests which have beenabandoned by their original contributor and are available for someone to pick up wherethe original contributor left off. The list of issues with the help wanted tag can befoundhere.Note that not all issues which need contributors will have this tag.

Building the documentation#

Before submitting a pull request check if your modifications have introducednew sphinx warnings by building the documentation locally and try to fix them.

First, make sure you haveproperly installed thedevelopment version. On top of that, building the documentation requires installing someadditional packages:

pipinstallsphinxsphinx-gallerynumpydocmatplotlibPillowpandas\polarsscikit-imagepackagingseabornsphinx-prompt\sphinxext-opengraphsphinx-copybuttonplotlypooch\pydata-sphinx-themesphinxcontrib-sasssphinx-design\sphinx-remove-toctrees

To build the documentation, you need to be in thedoc folder:

cddoc

In the vast majority of cases, you only need to generate the web site withoutthe example gallery:

make

The documentation will be generated in the_build/html/stable directoryand are viewable in a web browser, for instance by opening the local_build/html/stable/index.html file.To also generate the example gallery you can use:

makehtml

This will run all the examples, which takes a while. You can also run only a few examples based on their file names.Here is a way to run all examples with filenames containingplot_calibration:

EXAMPLES_PATTERN="plot_calibration"makehtml

You can use regular expressions for more advanced use cases.

Set the environment variableNO_MATHJAX=1 if you intend to view the documentation inan offline setting. To build the PDF manual, run:

makelatexpdf

Generated documentation on GitHub Actions#

When you change the documentation in a pull request, GitHub Actions automaticallybuilds it. To view the documentation generated by GitHub Actions, simply go to thebottom of your PR page, look for the item “Check the rendered docs here!” andclick on ‘details’ next to it:

Deprecation#

If any publicly accessible class, function, method, attribute or parameter is renamed,we still support the old one for two releases and issue a deprecation warning when it iscalled, passed, or accessed.

Deprecating a class or a function

Suppose the functionzero_one is renamed tozero_one_loss, we add the decoratorutils.deprecated tozero_one and callzero_one_loss from thatfunction:

from..utilsimportdeprecateddefzero_one_loss(y_true,y_pred,normalize=True):# actual implementationpass@deprecated("Function `zero_one` was renamed to `zero_one_loss` in 0.13 and will be ""removed in 0.15. Default behavior is changed from `normalize=False` to ""`normalize=True`")defzero_one(y_true,y_pred,normalize=False):returnzero_one_loss(y_true,y_pred,normalize)

One also needs to movezero_one fromAPI_REFERENCE toDEPRECATED_API_REFERENCE and addzero_one_loss toAPI_REFERENCE in thedoc/api_reference.py file to reflect the changes inAPI Reference.

Deprecating an attribute or a method

If an attribute or a method is to be deprecated, use the decoratordeprecated on the property. Please note that thedeprecated decorator should be placed before theproperty decoratorif there is one, so that the docstrings can be rendered properly. For instance, renamingan attributelabels_ toclasses_ can be done as:

@deprecated("Attribute `labels_` was deprecated in 0.13 and will be removed in 0.15. Use ""`classes_` instead")@propertydeflabels_(self):returnself.classes_

Deprecating a parameter

If a parameter has to be deprecated, aFutureWarning warning must be raisedmanually. In the following example,k is deprecated and renamed to n_clusters:

importwarningsdefexample_function(n_clusters=8,k="deprecated"):ifk!="deprecated":warnings.warn("`k` was renamed to `n_clusters` in 0.13 and will be removed in 0.15",FutureWarning,)n_clusters=k

When the change is in a class, we validate and raise warning infit:

importwarningsclassExampleEstimator(BaseEstimator):def__init__(self,n_clusters=8,k='deprecated'):self.n_clusters=n_clustersself.k=kdeffit(self,X,y):ifself.k!="deprecated":warnings.warn("`k` was renamed to `n_clusters` in 0.13 and will be removed in 0.15.",FutureWarning,)self._n_clusters=self.kelse:self._n_clusters=self.n_clusters

As in these examples, the warning message should always give both theversion in which the deprecation happened and the version in which theold behavior will be removed. If the deprecation happened in version0.x-dev, the message should say deprecation occurred in version 0.x andthe removal will be in 0.(x+2), so that users will have enough time toadapt their code to the new behaviour. For example, if the deprecation happenedin version 0.18-dev, the message should say it happened in version 0.18and the old behavior will be removed in version 0.20.

The warning message should also include a brief explanation of the change and pointusers to an alternative.

In addition, a deprecation note should be added in the docstring, recalling thesame information as the deprecation warning as explained above. Use the..deprecated:: directive:

..deprecated:: 0.13``k`` was renamed to``n_clusters`` in version 0.13 and will be removed   in 0.15.

What’s more, a deprecation requires a test which ensures that the warning israised in relevant cases but not in other cases. The warning should be caughtin all other tests (using e.g.,@pytest.mark.filterwarnings),and there should be no warning in the examples.

Change the default value of a parameter#

If the default value of a parameter needs to be changed, please replace thedefault value with a specific value (e.g.,"warn") and raiseFutureWarning when users are using the default value. The followingexample assumes that the current version is 0.20 and that we change thedefault value ofn_clusters from 5 (old default for 0.20) to 10(new default for 0.22):

importwarningsdefexample_function(n_clusters="warn"):ifn_clusters=="warn":warnings.warn("The default value of `n_clusters` will change from 5 to 10 in 0.22.",FutureWarning,)n_clusters=5

When the change is in a class, we validate and raise warning infit:

importwarningsclassExampleEstimator:def__init__(self,n_clusters="warn"):self.n_clusters=n_clustersdeffit(self,X,y):ifself.n_clusters=="warn":warnings.warn("The default value of `n_clusters` will change from 5 to 10 in 0.22.",FutureWarning,)self._n_clusters=5

Similar to deprecations, the warning message should always give both theversion in which the change happened and the version in which the old behaviorwill be removed.

The parameter description in the docstring needs to be updated accordingly by addingaversionchanged directive with the old and new default value, pointing to theversion when the change will be effective:

..versionchanged:: 0.22   The default value for`n_clusters` will change from 5 to 10 in version 0.22.

Finally, we need a test which ensures that the warning is raised in relevant cases butnot in other cases. The warning should be caught in all other tests(using e.g.,@pytest.mark.filterwarnings), and there should be no warningin the examples.