Everything you ever wanted to know about Linux -stable releases

Rules on what kind of patches are accepted, and which ones are not, into the“-stable” tree:

  • It or an equivalent fix must already exist in Linux mainline (upstream).

  • It must be obviously correct and tested.

  • It cannot be bigger than 100 lines, with context.

  • It must follow theDocumentation/process/submitting-patches.rstrules.

  • It must either fix a real bug that bothers people or just add a device ID.To elaborate on the former:

    • It fixes a problem like an oops, a hang, data corruption, a real securityissue, a hardware quirk, a build error (but not for things markedCONFIG_BROKEN), or some “oh, that’s not good” issue.

    • Serious issues as reported by a user of a distribution kernel may alsobe considered if they fix a notable performance or interactivity issue.As these fixes are not as obvious and have a higher risk of a subtleregression they should only be submitted by a distribution kernelmaintainer and include an addendum linking to a bugzilla entry if itexists and additional information on the user-visible impact.

    • No “This could be a problem...” type of things like a “theoretical racecondition”, unless an explanation of how the bug can be exploited is alsoprovided.

    • No “trivial” fixes without benefit for users (spelling changes, whitespacecleanups, etc).

Procedure for submitting patches to the -stable tree

Note

Security patches should not be handled (solely) by the -stable reviewprocess but should follow the procedures inDocumentation/process/security-bugs.rst.

There are three options to submit a change to -stable trees:

  1. Add a ‘stable tag’ to the description of a patch you then submit formainline inclusion.

  2. Ask the stable team to pick up a patch already mainlined.

  3. Submit a patch to the stable team that is equivalent to a change alreadymainlined.

The sections below describe each of the options in more detail.

Option 1 isstrongly preferred, it is the easiest and most common.Option 2 is mainly meant for changes where backporting was not consideredat the time of submission.Option 3 is an alternative to the two earlieroptions for cases where a mainlined patch needs adjustments to apply in olderseries (for example due to API changes).

When using option 2 or 3 you can ask for your change to be included in specificstable series. When doing so, ensure the fix or an equivalent is applicable,submitted, or already present in all newer stable trees still supported. This ismeant to prevent regressions that users might later encounter on updating, ife.g. a fix merged for 5.19-rc1 would be backported to 5.10.y, but not to 5.15.y.

Option 1

To have a patch you submit for mainline inclusion later automatically picked upfor stable trees, add this tag in the sign-off area:

Cc: stable@vger.kernel.org

UseCc:stable@kernel.org instead when fixing unpublished vulnerabilities:it reduces the chance of accidentally exposing the fix to the public by way of‘git send-email’, as mails sent to that address are not delivered anywhere.

Once the patch is mainlined it will be applied to the stable tree withoutanything else needing to be done by the author or subsystem maintainer.

To send additional instructions to the stable team, use a shell-style inlinecomment to pass arbitrary or predefined notes:

  • Specify any additional patch prerequisites for cherry picking:

    Cc: <stable@vger.kernel.org> # 3.3.x: a1f84a3: sched: Check for idleCc: <stable@vger.kernel.org> # 3.3.x: 1b9508f: sched: Rate-limit newidleCc: <stable@vger.kernel.org> # 3.3.x: fd21073: sched: Fix affinity logicCc: <stable@vger.kernel.org> # 3.3.xSigned-off-by: Ingo Molnar <mingo@elte.hu>

    The tag sequence has the meaning of:

    git cherry-pick a1f84a3git cherry-pick 1b9508fgit cherry-pick fd21073git cherry-pick <this commit>

    Note that for a patch series, you do not have to list as prerequisites thepatches present in the series itself. For example, if you have the followingpatch series:

    patch1patch2

    where patch2 depends on patch1, you do not have to list patch1 asprerequisite of patch2 if you have already marked patch1 for stableinclusion.

  • Point out kernel version prerequisites:

    Cc: <stable@vger.kernel.org> # 3.3.x

    The tag has the meaning of:

    git cherry-pick <this commit>

    For each “-stable” tree starting with the specified version.

    Note, such tagging is unnecessary if the stable team can derive theappropriate versions from Fixes: tags.

  • Delay pick up of patches:

    Cc: <stable@vger.kernel.org> # after -rc3

  • Point out known problems:

    Cc: <stable@vger.kernel.org> # see patch description, needs adjustments for <= 6.3

There furthermore is a variant of the stable tag you can use to make the stableteam’s backporting tools (e.g AUTOSEL or scripts that look for commitscontaining a ‘Fixes:’ tag) ignore a change:

Cc: <stable+noautosel@kernel.org> # reason goes here, and must be present

Option 2

If the patch already has been merged to mainline, send an email tostable@vger.kernel.org containing the subject of the patch, the commit ID,why you think it should be applied, and what kernel versions you wish it tobe applied to.

Option 3

Send the patch, after verifying that it follows the above rules, tostable@vger.kernel.org and mention the kernel versions you wish it to be appliedto. When doing so, you must note the upstream commit ID in the changelog of yoursubmission with a separate line above the commit text, like this:

commit <sha1> upstream.

Or alternatively:

[ Upstream commit <sha1> ]

If the submitted patch deviates from the original upstream patch (for examplebecause it had to be adjusted for the older API), this must be very clearlydocumented and justified in the patch description.

Following the submission

The sender will receive an ACK when the patch has been accepted into thequeue, or a NAK if the patch is rejected. This response might take a fewdays, according to the schedules of the stable team members.

If accepted, the patch will be added to the -stable queue, for review by otherdevelopers and by the relevant subsystem maintainer.

Review cycle

  • When the -stable maintainers decide for a review cycle, the patches will besent to the review committee, and the maintainer of the affected area ofthe patch (unless the submitter is the maintainer of the area) and CC: tothe linux-kernel mailing list.

  • The review committee has 48 hours in which to ACK or NAK the patch.

  • If the patch is rejected by a member of the committee, or linux-kernelmembers object to the patch, bringing up issues that the maintainers andmembers did not realize, the patch will be dropped from the queue.

  • The ACKed patches will be posted again as part of release candidate (-rc)to be tested by developers and testers.

  • Usually only one -rc release is made, however if there are any outstandingissues, some patches may be modified or dropped or additional patches maybe queued. Additional -rc releases are then released and tested until noissues are found.

  • Responding to the -rc releases can be done on the mailing list by sendinga “Tested-by:” email with any testing information desired. The “Tested-by:”tags will be collected and added to the release commit.

  • At the end of the review cycle, the new -stable release will be releasedcontaining all the queued and tested patches.

  • Security patches will be accepted into the -stable tree directly from thesecurity kernel team, and not go through the normal review cycle.Contact the kernel security team for more details on this procedure.

Trees

Review committee

  • This is made up of a number of kernel developers who have volunteered forthis task, and a few that haven’t.