If you're encountering difficulties with Config Sync, this page introducesyou to some common tools and procedures that can help you identify and resolveproblems that you experience.

Upgrade to a supported version

ConsiderupgradingConfig Sync to asupportedversion.Upgrading often resolves common problems and gives you access to the mostcurrent functionalities.

Use thenomos command-line tool

Thenomos command-line tool providesessential insights into your Config Sync setup. The commands described inthe following sections are particularly helpful when you're trying to determinethe source of your problem or when you need to work with Cloud Customer Care.

View Config Sync status

Thenomos statuscommand provides you with aggregated data and errors to help you understandwhat's happening with your Config Sync installation. The followinginformation is available withnomos status:

• Installation status per cluster
• Syncing errors (both reading from Git as well as reconciling the changes)

Create a bug report

If you have a problem with Config Sync that requires help fromCloud Customer Care, you can provide them with valuable debugginginformation by using thenomos bugreport command.

This command generates a timestamped zip file with information on the Kubernetescluster set in yourkubectl context. The file also contains logs fromConfig Sync Pods. It doesn't contain information from the resources syncedwith Config Sync.

View the overview dashboard

TheConfig Sync dashboardprovides you with an overview of the status of the packages that Config Syncmanages and the status of the resources in these packages. Exploring thisdashboard can help you to get a quick overview of the status of yourConfig Sync installation and discover any packages that have issues.

• To access the dashboard, in the Google Cloud console go to theConfigpage in theFeatures section:

  Go to Config

Use monitoring and log analysis

Monitoring Config Sync and exploring its logs can help you determine thesource of bugs and to better understand any unexpected behavior.

Understand Config Sync metrics

UseConfig Sync metrics togain visibility into the health of Config Sync.

Monitor RootSync and RepoSync objects

When you install Config Sync using the Google Cloud console orGoogle Cloud CLI, Config Sync automatically creates aRootSync object foryou. When youConfigure syncing from multiple repositories,you can createRepoSync objects that contain configuration information about yournamespace repositories.

Monitoring these objects can reveal valuable information about the state ofConfig Sync. To learn more, seeMonitor RootSync and RepoSync objects.

Use service level indicators (SLIs)

To receive notifications when Config Sync isn't working as intended,use Config Sync SLIs.

Query logs

You can use theLogs Explorerto retrieve, view, and analyze log data for Config Sync. These logs cancontain valuable historical data that isn't captured bynomos bugreport whenthe operator or reconciler Pods are restarted. For examples of queries thatmight help you diagnose your issue, seeQuery Config Sync logs.

Examine resources with thekubectl command-line tool

Config Sync is composed of multiple custom resources that you can query byusingkubectl commands. These commands help you understand the status of eachof Config Sync's objects.

You should know the following information about the Kubernetes resources thatConfig Sync manages:

• config-management-system is the namespace we use to run all coresystem components of Config Sync.
• configmanagement.gke.io andconfigsync.gke.io are the API groups weuse for all custom resources.

Examples

The following sections show you how you might usekubectl commands to examineConfig Sync.

List custom resources

• You can get a full list of the custom resources by running the followingcommand:

  kubectlapi-resources|grep-E"configmanagement.gke.io|configsync.gke.io"

• Individual custom resources can be consumed by running the following command:

  kubectlgetRESOURCE-oyaml.

  ReplaceRESOURCE with the name of the resource that youwant to query.

  For example, the output of the following command lets you check the status of aRootSync object:

  kubectlgetrootsync-nconfig-management-system-oyaml

Check an object's token annotation

You might want to know when a managed Kubernetes object was last updated byConfig Sync. Each managed object is annotated with the hash of the Gitcommit when it was last modified, and the path to the config that contained themodification.

For example, to get the annotation of a ClusterRoleBinding namednamespace-readers, run the following command:

kubectlgetclusterrolebindingnamespace-readers

The output is similar to the following:

apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRoleBindingmetadata:  annotations:    configmanagement.gke.io/source-path: cluster/namespace-reader-clusterrolebinding.yamlconfigmanagement.gke.io/token: bbb6a1e2f3db692b17201da028daff0d38797771  name: namespace-readers...

For more information, seelabels and annotations.

Accelerate diagnosis with Gemini Cloud Assist

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of theService Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see thelaunch stage descriptions.

Sometimes, the cause of your issue isn't immediately obvious, even after usingthe tools discussed in the preceding sections. Investigating complex cases canbe time-consuming and requires deep expertise. For scenarios like this,Gemini Cloud Assist can help. It can automatically detect hiddenpatterns, surface anomalies, and provide summaries to help you quickly pinpointa likely cause.

As an early-stage technology, Gemini for Google Cloud products can generate output that seems plausible but is factually incorrect. We recommend that you validate all output from Gemini for Google Cloud products before you use it. For more information, seeGemini for Google Cloud and responsible AI.

Access Gemini Cloud Assist

To access Gemini Cloud Assist, complete the following steps:

1. In the Google Cloud console, go to any page.

2. In the Google Cloud console toolbar, clickOpen or close Gemini Cloud Assist chat.

   TheCloud Assist panel opens. You can click example prompts if they aredisplayed, or you can enter a prompt in theEnter a prompt field.

Explore example prompts

To help you understand how Gemini Cloud Assist can help you, hereare some example prompts:

For more information, see the following resources:

• Learn how towrite better prompts.
• Learn how to use theGemini Cloud Assist panel.
• Read theGemini for Google Cloud overview.
• Learnhow Gemini for Google Cloud uses your data.

Use Gemini Cloud Assist Investigations

In addition to interactive chat, Gemini Cloud Assist can perform moreautomated, in-depth analysis through Gemini Cloud AssistInvestigations. This feature is integrated directly into workflows likeLogs Explorer, and is a powerful root-cause analysis tool.

When you initiate an investigation from an error or a specific resource,Gemini Cloud Assist analyzes logs, configurations, and metrics.It uses this data to produce ranked observations and hypotheses about probableroot causes, and then provides you with recommended next steps. You can alsotransfer these results to a Google Cloud support case to provide valuablecontext that can help you resolve your issue faster.

For more information, seeGemini Cloud Assist Investigationsin the Gemini documentation.

Read additional troubleshooting documentation

If you're still experiencing problems, the following resources might behelpful:

• If you've received an error message, see theerrorreference page for advice on resolving theerror.

• Check to see if the problem you're having is caused by aknown issue.

• If you're having difficulties with a specific area, one of the targetedtroubleshooting guides listed in theTroubleshoot by issue type section ofthe table of contents might help.

What's next

• If you can't find a solution to your problem in the documentation, seeGetsupport for further help, including advice on the following topics:

  • Opening a support case by contactingCloud Customer Care.
  • Getting support from the community byasking questions onStackOverflow.If you use kpt or Kustomize, use thekpt orkustomize tag tosearch for similar issues.
  • Opening bugs or feature requests by using thepublic issue tracker on GitHub.