You can usePatchto apply operating system patches across a group of virtual machine (VM)instances.

To apply patches to your VMs, complete the following steps:

1. Set up your VM.
2. Run a patch job.

Before you begin

• ReviewOS Config quotas.
• To generate audit logs for VM Manager events,enable data access audit logs.
• If you haven't already, set upauthentication. Authentication verifies your identity for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:

  Select the tab for how you plan to use the samples on this page:

  Console

  When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.

  gcloud

  1. Install the Google Cloud CLI. After installation,initialize the Google Cloud CLI by running the following command:

     gcloudinit

     If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

     Note: If you installed the gcloud CLI previously, make sure you have the latest version by runninggcloud components update.
  2. Set a default region and zone.

  REST

  To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.

  Install the Google Cloud CLI. After installation,initialize the Google Cloud CLI by running the following command:

  gcloudinit

  If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

  Note: If you installed the gcloud CLI previously, make sure you have the latest version by runninggcloud components update.

  For more information, seeAuthenticate for using REST in the Google Cloud authentication documentation.

Limitations

• You can deploy and run patch jobs only for VMs in a single Google Cloud project.You cannot run patch jobs across Google Cloud projects, evenif the VMs are in a Shared VPC. However, you canview the patch compliancedata across projects.
• By default, VM Manager does not patch any VMs that are part ofa managed instance group (MIG). You can override this default behavior when you create the patch job.The following limitations apply when you patch VMs that are part of a MIG:
  • When aMIG repairs a VM,it recreates the VM based on the instance template. This might revert theVM to an unpatched state.
  • Patching VMs can cause unexpected results in a MIG that has autoscalingenabled.Autoscaler deletes patched VMs when the load goes down and creates new VMs,without any patches, using MIG's instance template when the load increases.For example, if the average CPU utilization is less than the targetutilization you specified for autoscaling, MIG can remove some of the patchedVMs while scaling in.

Supported operating systems

For the full list of operating systems and versions that supportPatch, seeOperating system details.

Set up your VM

To use the Patch feature, complete the following steps:

1. For all VMs,set up VM Manager.
2. For Windows VMs, Google recommendsdisabling automatic updates on the VMs. This reduces conflicts between the Windows automatic updates andthe Patch service.

Permissions

Owners of a Google Cloud project have full access to run and manage patch jobs.For all other users, you need to grant permissions. You can grant one of thefollowing granular roles:

• roles/osconfig.patchJobExecutor: Contains permissions to run, cancel, get,and list patch jobs. It also contains permissions to view instance details fora patch job.
• roles/osconfig.patchJobViewer: Contains permissions for read-only access toget and list patch jobs. It also contains permissions to view instance detailsfor a patch job.

For example, to grant a user access to run patch jobs, use the followingcommand:

gcloud projects add-iam-policy-bindingproject-id \    --member user:user-id@gmail.com \    --role roles/osconfig.patchJobExecutor

Replace the following:

• project-id: The project ID.
• user-id: The user's Google Workspace username.

Run patch jobs

You can run a patch job by using either theGoogle Cloud console,Google Cloud CLI, orREST.

When you run a patch job, the patching of the VMs starts simultaneouslyon all instances specified by theinstance filter.

After you have started a patch job, you can monitor your patches using thePatch dashboard.It takes approximately 30 minutes after a patch job starts before the data ispopulated on the dashboard.

gcloud compute os-config patch-jobs executeinstance-filter

gcloud compute os-config patch-jobs execute \    --instance-filter-names="zones/us-east1-b/instances/instance-1" \    --description "patch for instance-1"

gcloud compute os-config patch-jobs execute \    --instance-filter-all \    --duration="1h30m" --reboot-config="DEFAULT" \    --apt-dist --windows-exclusive-patches=4339284 \    --yum-minimal --yum-security \    --async

POST https://osconfig.googleapis.com/v1/projects/project-id/patchJobs:execute{  "instanceFilter":instance-filter}

POST https://osconfig.googleapis.com/v1/projects/project-id/patchJobs:execute{  "description":"patch instance1 in us-east1-b",  "duration":"5400s",  "instanceFilter":{    "instances":[      "zones/us-east1-b/instances/instance1"    ]  }}

POST https://osconfig.googleapis.com/v1/projects/project-id/patchJobs:execute{  "instanceFilter":{    "groupLabels":[      {        "labels":{          "env":"dev",          "app":"web"        }      }    ],    "instanceNamePrefixes":[      "test-"    ],    "zones":[      "asia-east1-b",      "asia-east1-c"    ]  }}

POST https://osconfig.googleapis.com/v1/projects/project-id/patchJobs:execute{ "duration":"5400s", "instanceFilter":{   "all":true }, "patchConfig":{   "rebootConfig":"DEFAULT",   "apt":{     "type":"DIST"   },   "yum":{     "security":true,     "minimal":true   },   "windowsUpdate":{     "exclusivePatches":"4339284"   } }}

Instance filters

You can specify the instances to be included in a patch job by usingfilters. The following filters are supported for patch jobs:

• Filter by name: Limit patch job to instances with specific names.Instance names must be specified by using the full URI. SupportedURI formats include the following:

  • zones/zone/instances/instance-name

  • projects/project-id/zones/zone/instances/instance-name

  • https://www.googleapis.com/compute/v1/projects/project-id/zones/zone/instances/instance-name

• Filter by name prefix: Limit patch job to instances with a specific prefixin their name.

• Filter by zone: Limit patch job to instances in a specific zone.

• Filter by label: Limit patch job to instances with specific labels.

You can also run patch jobs on all instances in a Google Cloud project by setting theall field ininstanceFilter totrue. Formore information, seeexample instance filters.

--instance-filter-all

{  "instanceFilter":{    "all":"true"  }}

--instance-filter-names="zones/us-east1-b/instances/instance1"

{  "instanceFilter":{    "instances":[      "zones/us-east1-b/instances/instance1"    ]  }}

--instance-filter-name-prefixes="app-"

{  "instanceFilter":{    "instanceNamePrefixes":[      "app-"    ]  }}

--instance-filter-zones="us-east1-b","us-east1-c"

{  "instanceFilter":{    "zones":[      "us-east1-b",      "us-east1-c"    ]  }}

--instance-filter-group-labels="env=dev,app=web"--instance-filter-group-labels="env=dev,app=worker"

{  "instanceFilter":{    "groupLabels":[      {        "labels":{          "env":"dev",          "app":"web"        }      },      {        "labels":{          "env":"dev",          "app":"worker"        }      }    ]  }}

gcloud compute os-config patch-jobs execute \    --instance-filter-name-prefixes="test-" \    --instance-filter-zones="us-east1-c" \    --instance-filter-group-labels="env=prod,app=web"

• For Windows, you specify the classification of patches to apply(eg.Security andCritical) or target specific KBs to exclude.For more information about patch classification,see theMicrosoft support documentation.

• For RHEL, Rocky Linux, and CentOS, the underlying system isyum.

  • For patches that target RHEL and Rocky Linux VMs, you can specifysecurity andminimal packages.
  • For CentOS VMs, there is nosecurity metadata in theCentOSyum repository. Therefore, you need not specify thesecurity option when updating security packages. If you don't specifyany package, the patch job updates allpackages, including those with security updates.
  • You can also exclude specific packages. For more information, see theyum man pages.

• For Debian & Ubuntu, the underlying system isapt. For patches that targetthese VMs, you can specifydist-upgrade or a standard upgrade. You canalso exclude specific packages.For more information, see either theDebian man pages orUbuntu man pages.

• For SuSE, the underlying system iszypper, specifically usingzypper patches.For patches that target these VMs, you can specify options such as:

  • with update: update all packages not covered by patches
  • with optional: optional patches are treated as needed
  • The categories or severities of patches to apply

  You can also exclude specific patches.

Optionally, for all the supported operating systems, you can select to installapproved patches only by specifying these updates. This lets youenter a list of approved packages or patches. When you select these approved patches,only the approved packages or patches are installed. All other patch configurationparameters are skipped during the update.

Examples

gcloud compute os-config patch-jobs execute \    --instance-filter-zones="northamerica-northeast1-a" \    --apt-dist \    --yum-security \    --yum-minimal \    --zypper-categories=security \    --windows-classifications=critical,security \    --reboot-config=default \    --skip-unpatchable-vms

gcloud compute os-config patch-jobs execute --help

POST https://osconfig.googleapis.com/v1/projects/project-id/patchJobs:execute{    "instanceFilter":{        "zones":[            "northamerica-northeast1-a"        ]    },    "patchConfig":{        "apt": {            "type": "dist-upgrade"        },        "yum": {            "security": true,            "minimal": true        },        "zypper": {            "categories": ["security"]        },        "windowsUpdate": {            "classifications": ["CRITICAL", "SECURITY"]        },        "rebootConfig": "DEFAULT"    },    "skipUnpatchableVms": true}

Settings for VMs in MIGs and unpatchable VMs

To specify how VM Manager handles VMs that are part of a managed instance group (MIG),or VMs that run operating systems that don't support patching, use the following boolean options in thepatch configuration:

• Allow patching of MIG instances: By default, VM Manager does notpatch any VMs that are part of a MIG. If you want to patch VMs in MIGs,setmigInstancesAllowed totrue. For more information about limitationsfor patching VMs in a MIG, seeLimitations.

• Skip unpatchable VMs: If you setskipUnpatchableVms totrue,VM Manager skips patching a VM if any of the following conditionsare met:

  • The VM's operating system does not support patching. For example,Container-Optimized OS. SeeSupported operating systems.
  • The VM is part of a MIG, andmigInstancesAllowed is set tofalse.

  IfskipUnpatchableVms is set totrue, the patch job reports the patch statusof unpatchable VMs asSKIPPED. Additionally, if no other errors occur,the patch job reports a state ofCOMPLETED_WITH_INACTIVE_VMS instead ofCOMPLETED_WITH_ERRORS for inactive VMs.

Maintenance window

A maintenance window is the total length of time that you allow a patch job to run.Patch jobs will timeout if they don't complete within the specified maintenancewindow.

For example, if you set a maintenance window of60 minutes, then no new patchjobs will be initiated 60 minutes after the start time. Some processes such asdownloading a file or rebooting might occur outside of this maintenance window,however no new patch jobs will be initiated.

Reboot options

When running a patch job, you can specify the reboot options for the patch. Thefollowing options are available:

• Default: The agent decides if a reboot is necessary by checking well knownsignals on each OS. Multiple reboots may occur during the patching and mayoccur before any patches are installed.
• Always: The machine reboots after the update has completed.
• Never: The machine does not reboot after the update has completed. In somecases, this might mean that not all patches are fully applied.

Pre-patch and post-patch scripts

When running a patch job, you can specify scripts to be run as part of thepatching process. These scripts are useful for performing tasks such as shuttingdown an application and performing health checks.

• Pre-patch scripts run before patching starts. If a system reboot isrequired before patching starts, the pre-patch script runs before thereboot.
• Post-patch scripts run after patching completes. If a system reboot isrequired as part of the patching, the post-patch script runs after the reboot.

A patch job accepts one pre-patch and one post-patch script for Linux, andone pre-patch and one post-patch script for Windows. Linux and Windows scriptsmust be provided using the appropriate flags, parameters, or sections when specifiedfrom theGoogle Cloud CLI,REST, or theGoogle Cloud console respectively. Linux scripts run on LinuxVMs only and Windows scripts run on Windows VMs only.

These script files can either be stored on the VM or in aversioned Cloud Storage bucket.

Store patch scripts in Cloud Storage buckets

If you want to use a Cloud Storage bucket to store your scripts,create a Cloud Storage bucketandupload your scripts to the bucket.When using a Cloud Storage bucket, consider the following:

• If your Cloud Storage object is not publicly readable, ensure thatthe service account attached to the instancehas the necessary IAM permissions to read the Cloud Storageobjects. To ensure that you have the correct permissions, check thepermission settings on theCloud Storage object.
• When you select a script from Cloud Storage using theGoogle Cloud console, the latest version of the specified Cloud Storage objectis used by default.

• If your organization enforces theResource Location Restriction constraint,then you must store your scripts in Cloud Storage buckets in regions and zonesthat your org policy permits.

gcloud compute os-config patch-jobs execute \    --instance-filter-zones="northamerica-northeast1-a" \    --async \    --pre-patch-linux-executable="/tmp/pre_patch_script.sh" \    --post-patch-linux-executable="gs://my-patch-scripts/linux/post_patch_script#1523477886880" \    --pre-patch-windows-executable="C:\\Users\\user\\pre-patch-script.cmd" \    --post-patch-windows-executable="gs://my-patch-scripts/windows/post_patch_script.ps1#135920493447"

gcloud compute os-config patch-jobs execute --help

POST https://osconfig.googleapis.com/v1/projects/project-id/patchJobs:execute{  "instanceFilter":{    "zones":[      "northamerica-northeast1-a"    ]  },  "patchConfig":{    "preStep":{      "linuxExecStepConfig":{        "localPath":"/tmp/pre_patch_script.sh"      },      "windowsExecStepConfig":{        "interpreter":"SHELL",        "localPath":"C:\\Users\\user\\pre-patch-script.cmd"      }    },    "postStep":{      "linuxExecStepConfig":{        "gcsObject":{          "bucket":"my-patch-scripts",          "generationNumber":"1523477886880",          "object":"linux/post_patch_script"        }      },      "windowsExecStepConfig":{        "gcsObject":{          "bucket":"my-patch-scripts",          "generationNumber":"135920493447",          "object":"windows/post_patch_script.ps1"        },        "interpreter":"POWERSHELL"      }    }  }}

Patch rollout options

You can choose to either patch VMs one zone at a time (zone by zone), or to patchall zones at once (concurrent zones).

Along with making a selection for the zone roll out, you can also specify azone disruption budget for your VMs.

Zone disruption budget

A disruption budget is the maximum number (or percentage) of VMs per zone todisrupt at any given moment.

What is considered a disrupted VM?

During patching, a VM is considered disrupted from the time the OS Config agentis notified to begin until patching has completed. This disruption time includesthe time to complete reboot and any post-patch steps.

A VM is also counted as a part of the disruption budget if it meets any of thefollowing conditions:

• Patching operation fails when applying the patches
• Patching operation fails when running pre or post patch steps
• Patching operation fails to respond with a success notification before timingout

gcloud compute os-config patch-jobs execute \   --instance-filter-all \   --rollout-mode=zone-by-zone \   --rollout-disruption-budget=10

gcloud compute os-config patch-jobs execute \   --instance-filter-all \   --rollout-mode=concurrent-zones \   --rollout-disruption-budget-percent=50

POST https://osconfig.googleapis.com/v1/projects/project-id/patchJobs:execute{  "instanceFilter":{    "zones":[      "us-central1-a",      "us-central1-c",      "us-central1-f"    ]  },  "rollout": {    "disruptionBudget": {      "percent": 25    },    "mode": "CONCURRENT_ZONES"  }}

Enable patching Microsoft software on Windows VMs

When you run a patch job on Windows VMs, by default, Patchapplies only the patches for the Windows operating system.

You can apply updates for Microsoft software such as Microsoft SQL Server,SharePoint Server, or .NET framework running on yourWindows VMs when you run a patch job. By default, patching theseapplications is disabled to avoid disruption of service and to separate plannedupdates for these software. To enable patching Microsoft software automatically,you can either use the Windows UI or the PowerShell.

 $service_manager = New-Object -ComObject 'Microsoft.Update.ServiceManager' $service_manager.AddService2("7971f918-a847-4430-9279-4a52d1efe18d",7,"")

Debug a patch job

If your patch fails, you can use the following steps to help find and resolvethe issues.

1. Review theinstance detailsfor the affected patch job. This helps you to identify which instancesfailed or what state they are stuck in. The instance details list alsocontains a brief error message for each instance.

   If a patch fails with a status ofNO_AGENT_DETECTED orTIMED_OUT, this usually meansthat the service sent a request to the agent to beginpatching but never heard back from the agent. Review the following possiblecauses and fixes:

   • The instance is not running. To fix this,start the VM instance.
   • Verify the setup by using theverification checklist.
   • The network settings on the VPC network or theinstance didn't allow the OS Config agent to communicate with theOS Config API. To fix this, check the network settings.

2. If theinstance detailsdon't provide enough information, review the Cloud Logging logsorserial port console.The OS Config agent writes its log entries to both locations.In Cloud Logging, you can filter using the patch job ID to see alllog entries related to that patch job. You can also enable debug logging bysetting theosconfig-log-level=debugmetadata valueat the VM or Google Cloud project level.

What's next?

• Learn more aboutPatch.
• Manage your patch jobs.
• Schedule patch jobs.