Linux kernel live patching is a way to apply critical and important security patches to a running Linux kernel, without the need to reboot or interrupt runtime.

There is a difference between a patch and an update. An update is a new, more minor version of a package and it can contain bug fixes, performance improvements, new features, edits at the command line, and other enhancements. A patch is a partial snippet of code (usually the differing output between two versions of a package or file) that fixes a vulnerability in the existing version. These patches fix vulnerabilities without latency so that the existing implementation can run more securely and system administrators can hold off on rebooting until the next regular maintenance window.

In theory, live patching could be used for any patch for the running kernel, including regular bug fixes and enhancements. In practice, the priority function for live patching has been major security vulnerabilities, since these are fixes that shouldn’t be delayed. With Red Hat® Enterprise Linux®, we have focused on security patches and kernel live patching is considered a security feature (with management benefits for administrators).

Without live patching, applying a patch requires the reboot of the patched service or app in order to load the new changes of the patched version. This is true even for patches applied tothe Linux kernel itself, creating a conundrum for Linux system administrators: They need to apply imperative security updates, but that means unscheduled reboots to thelinux server, including latency and downtime.

This was the predicament encountered by a student system administrator at MIT, Jeff Arnold. He had a system with a pending security update and chose to delay addressing the vulnerability. That system ended up being hacked. In 2008, Arnold decided to address the challenge he’d faced by creating what he called"rootless kernel security updates" — a way of applying and loading security patches to a system as it was running. That initial project (ksplice) was the creation of the concept of Linux kernel live patching.

Dynamic software updates are a challenge in computing because of the complexity of dealing with things likestateless versus stateful services, persistence,data management, transactions, and following commands—all things that are at the very foundation of how a service or application operates.

What Arnold and subsequent projects for Linux kernel live patching did was use tools within the kernel space to load kernel modules.

Linux system operations are divided into two sections: the user space (where all the services and applications operate), and the kernel space (where the core system operations are). The kernel is an intermediary for all of the applications to access hardware resources like the CPU and storage. Along with the kernel itself, administrators can create custom kernel modules to extend or modify functionality, and these kernel modules can be loaded and executed dynamically, even after boot.

What kernel live patching tools do is create a kernel module out of the patched code, then using the ftrace (function trace) tool to route from the obsolete function to the new replacement function, patch module, or patched function.

Figure 1: How kernel live patching works

Ksplice was the first project for live patching the Linux kernel; however, ksplice was sold to Oracle and eventually changed to a closed-source tool. Otherdevelopment teams began trying to come up with open source projects that could replace ksplice, with two slightly different projects launching in 2014:kpatch from Red Hat and kgraft from SuSE. Ultimately, for the good of the Linux kernel community, Red Hat and SuSE developers worked together to create livepatch, which is a common layer within the Linux kernel that allows people to develop compatible kernel live patching tools.

The key thing to remember is that patches are specifically used for addressing security risks. One of the challenges that sysadmins have is that they not only need to account for security patching in Linux systems, they also need to make sure that they are meeting uptime requirements. This means they may not be able to take systems offline outside preset maintenance windows.  This is a constant tension that is only increasing.

According tothe Red Hat Security Risk Report, there was a massive spike of identified security issues (calledcommon vulnerabilities and exposures or CVEs) from 2019 to 2020—up about 155% (1,313 to 2,040). Although that dropped slightly in 2021 (1,596), it was still 22% higher than 2019. Most of this has been an increase in moderate severity vulnerabilities; important CVEs have stayed roughly steady and critical vulnerabilities have been dropping.

Kernel updates and patches are released every 6 weeks, and then minor updates of Red Hat Enterprise Linux come out every six months with all the security patches included. The ability for live patching service to be done to the Linux kernel without a reboot removes the need to decide which policy to follow.

More about determining your kernel version

CVEs related to the kernel are one of the top concerns, with four of the top 10 most-viewed CVEs in 2021 relating to the kernel.

Being able to perform Linux kernel security patching live is much more than a convenience. It is an important tool for security teams to proactively address security vulnerabilities, keep kernel functions running, and maintain the safety of their systems.

Read more about overcoming vulnerabilities with Linux security patching

Read more reviews

Kernel live patching is one specific tool to manage security, but it is far from the only one available withRed Hat Enterprise Linux. Red Hat Enterprise Linux includesdifferent tools for managing security, such asSELinux for permissions and access control and system roles to help automate configuration and tasks.Upgrading your Red Hat Enterprise Linux infrastructure can help maintain your security posture by ensuring uninterrupted access to the latest fixes. 

There is no single solution to create a "secure environment." Threats are always changing and evolving, with different attack vectors emerging and different targets pinpointed. Security starts at the operating system level—even the very source code—and works up through the technology stack and throughout the lifecycle, which is why having a variety of tools is essential.

Manage kernel live patching from theRed Hat Enterprise Linux web console to greatly reduce the complexity of performing critical maintenance. This new web console capability provides a simplified interface for both highly-skilled and non-experienced administrators to apply kernel updates without having to use command line tooling.

Red Hat Insights is available with any Red Hat Enterprise Linux subscription, providing a unified management experience for addressing security concerns. This includes:

• Visual dashboards that cover your entire infrastructure, showing vulnerable systems and patch status, allowing you to find and fix critical bugs without a separate subscription to Red Hat Satellite Server.
• Relevant CVEs and other updates.
• Playbooks for automated Linux security patching.
• Both defined security profiles and custom profiles to manage system configuration.
• Using baseline configs and flagging systems which diverge from the baselines.

Read more about security and compliance

Linux kernel live patching came to exist when a need was identified and an open project created to work collaboratively with organizations and community members, creating a solution that would benefit the community. This open way of working is at the core of Red Hat’svalue of subscription principles: collaboration, transparency, and a focus on improving the technology and user experience.  

Read more about Red Hat Enterprise Linux security features

• Blog post

  Red Hat Enterprise Linux's evolution at Summit: 12 essential reads

• Overview

  Modernize your IT infrastructure with Red Hat and AWS

• Blog post

  Now available: Red Hat Enterprise Linux Security Select Add-On

• Overview

  Modernize your IT infrastructure with Red Hat and AWS

• Why run Linux on AWS?
• What is high availability?
• What is ERP?
• Functional safety and continuous certification on Linux
• What is SAP HANA (and why does it run on Linux)?
• Why run Linux on IBM Cloud
• What is CentOS Stream?
• Why choose Red Hat Enterprise Linux on Azure?
• Why choose Red Hat Enterprise Linux on AWS?
• Linux on Azure
• Linux for cloud computing
• Red Hat Enterprise Linux migration process
• Why choose Red Hat for Linux?
• What to know about CentOS Linux EOL
• What to know for Red Hat Enterprise Linux 7 End of Maintenance
• What is CentOS?
• How to move to Red Hat Enterprise Linux from other Linux distros
• Why run Linux on Google Cloud?
• Red Hat Satellite on Red Hat Enterprise Linux
• Why choose Red Hat Enterprise Linux on Google Cloud?
• What is the SAP HANA migration?
• Why run Microsoft SQL Server on Linux?
• What's the best Linux distro for you?
• What's the difference between Fedora and Red Hat Enterprise Linux
• What is Linux?
• What is an image builder?
• What is a Linux server?
• What is a configuration file?
• What is high performance computing (HPC)?
• What is a golden image?
• ARM vs x86: What's the difference?
• What is an ARM processor?
• What is lightweight directory access protocol (LDAP) authentication?
• What is MPLS?
• Red Hat Enterprise Linux security
• What is a Linux certification?
• What was CoreOS and CoreOS container Linux
• What is SELinux?
• What is the Linux kernel?