Custom Resource Definitions (CRDs)are powerful tools forextending Kubernetescapabilities.However, if a CRD contains an invalid or malformed Certificate Authority (CA)bundle within its conversion webhook configurationspec.conversion.webhook.clientConfig.caBundle, it can disrupt clusteroperations. This issue can manifest as errors during resource creation, updates,or deletions, affecting the stability and performance of your cluster.

To prevent this issue, Google Kubernetes Engine (GKE) automatically detects CRDswith invalid CA bundles and generates arecommendation. Usethis document to find the recommendation, identify your misconfigured CRDs, andupdate them.

This information is important forPlatform admins and operatorsand other users who manage CRDs and custom resources in GKE.

Identify impacted clusters

To get insights identifying clusters that are affected by CRDs with invalid CA bundles, followthe instructions toview insights and recommendations for subtypeK8S_CRD_WITH_INVALID_CA_BUNDLE. You can get insights inthe following ways:

• Use the Google Cloud console.
• Use the Google Cloud CLI, or the Recommender API, filtering with the subtypeK8S_CRD_WITH_INVALID_CA_BUNDLE.

After you identify the CRDs using the insights, follow the instructions totroubleshoot the misconfigured CA bundle.

When GKE detects misconfigured CRDs

GKE generates an insight and recommendation with theK8S_CRD_WITH_INVALID_CA_BUNDLE subtype if the GKE cluster hasone or more CRDs reporting a misconfiguredcaBundle for the webhook clientconfiguration inspec.conversion.webhook.clientConfig.

Follow the instructions tocheck CRDs with misconfigured CA bundle.

Troubleshoot the detected CRDs

The following sections have instructions for you to troubleshoot the CRDs thatGKE detected as potentially misconfigured.

After you implement the instructions and the CRDs are correctly configured, therecommendation is resolved within 24 hours and no longer appears in the console.If it has been less than 24 hours since you've implemented the guidance of therecommendation, you canmark the recommendation asresolved.If you don't want to implement the recommendation, you candismissit.

Identify affected CRDs in a cluster

1. View insights andrecommendations for subtypeK8S_CRD_WITH_INVALID_CA_BUNDLE,choosing one insight at a time to troubleshoot. GKE generatesone insight per cluster which has a broken CRD.

2. Run the following command to describe the Service to find CRDs withpotentially problematic CA bundles:

   kubectlgetcrd-ocustom-columns=NAME:.metadata.name,CABUNDLE:.spec.conversion.webhook.clientConfig.caBundle

   The output includes the following:

   • Name: The name of the CRD.
   • CaBundle: The CA bundle associated with the CRD's conversionwebhook, if present. Examine the output. If the caBundle column is emptyfor a CRD that you know utilizes a conversion webhook, this signals apotential issue with the caBundle.

Recreate the CRD

To resolve this error, recreate the affected CRD with a valid CA bundle:

1. Back up existing custom resources associated with this problematicCRD, if you have any. Run the following command to export the existingresources:

   kubectlget<crd-name>-oyaml >backup.yaml

2. Delete the existing CRD:

   kubectldeletecrd<crd-name>

3. Ensure that thecaBundle field of the CRD contains a well-formed,base-64-encoded PEM certificate. You can do this either by editing theCRD directly or by reaching out to its authors.

4. Modify the CRD YAML definition, updating thespec.conversion.webhook.clientConfig.caBundle field with the validCA bundle data. The result should look something like the following:

   spec:conversion:webhook:clientConfig:caBundle:<base64-encoded-ca-bundle>

5. Apply the corrected CRD:

   kubectlapply-f<corrected-crd-file.yaml>

6. Restore your custom resources:

   kubectlapply-fbackup.yaml

What's next

• Optimize your usage of GKE with insights andrecommendations
• Troubleshooting common issues