- Notifications
You must be signed in to change notification settings - Fork165
Add troubleshooting topic for runtimes#1197
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Merged
Uh oh!
There was an error while loading.Please reload this page.
Merged
Changes fromall commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Uh oh!
There was an error while loading.Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading
Uh oh!
There was an error while loading.Please reload this page.
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,229 @@ | ||
| --- | ||
| title: "Troubleshooting GitOps Runtimes" | ||
| description: "Troubleshoot installation/communication issues for Runtimes" | ||
| --- | ||
| ## Pre-installation checks or runtime collision check failed | ||
| ### Possible cause | ||
| You have Argo Project components from other installations in your cluster. | ||
| ### Possible actions | ||
| You can either: | ||
| * Install Codefresh GitOps Runtime alongside Community Argo CD | ||
| _OR_ | ||
| 1. Uninstall the Codefresh GitOps Runtime. | ||
| 1. Remove the Argo Project components from your cluster. | ||
| 1. Reinstall the Codefresh GitOps Runtime. | ||
| ## `Job has reached the specified backoff limit` error during Helm installation | ||
| ### Possible cause | ||
| Validation errors in your `values.yaml` file. | ||
| Before initiating the installation, Codefresh automatically validates the `values.yaml` file to verify that the supplied values are correct. | ||
| ### Possible actions | ||
| 1. Get more detailed information on the reason for the validation failure by running: | ||
| `kubectl logs jobs/validate-values -n ${NAMESPACE}` | ||
| where: | ||
| `{NAMESPACE}` is the namespace of the Hybrid GitOps Runtime. | ||
| 1. Fix the errors. The table below describes the settings that are validated in the `values` file. | ||
| | Setting | Validation | | ||
| |----------------------|------------------------------------------------------------------------------------------------------------| | ||
| | `userToken` | If explicitly defined, or defined as a `secretKeyRef` which exists in the current k8s context and the defined namespace. | | ||
| | **Account permissions** | If the user has admin permissions for the account in which they are installing the runtime. | | ||
| | **Runtime name** | If defined, and is unique to the account. | | ||
| | **Access mode** | {::nomarkdown}<ul><li>For tunnel-based, the default, if `accountId` is defined, and matches the account of the `userToken` defined in the file.</li><li>For ingress-based, if the `hosts` array contains at least one entry that is a valid URL (successful HTTP GET).</li><li>If both tunnel-based and ingress-based access modes are disabled, if `runtime.ingressUrl` is defined.</li></ul>{:/} | | ||
| | `gitCredentials` | When defined, includes a Git password either explicitly, or as a `secretKeyRef`, similar to `userToken`. The password or token has the required permissions in the Git provider. | | ||
| ## Pre-installation error: please upgrade to the latest cli version: `v<number>` | ||
| ### Possible cause | ||
| Codefresh has a new CLI version. | ||
| ### Possible actions | ||
| Run the appropriate command to upgrade to the latest version: | ||
| {: .table .table-bordered .table-hover} | ||
| | Download mode | OS | Command | | ||
| | -------------- | ----------| ----------| | ||
| | `Curl` | MacOS-x64 | `curl -L --output - https://github.com/codefresh-io/cli-v2/releases/latest/download/cf-darwin-amd64.tar.gz | tar zx && mv ./cf-darwin-amd64 /usr/local/bin/cf && cf version`| | ||
| | | MacOS-m1 |`curl -L --output - https://github.com/codefresh-io/cli-v2/releases/latest/download/cf-darwin-arm64.tar.gz | tar zx && mv ./cf-darwin-arm64 /usr/local/bin/cf && cf version` | | ||
| | | Linux - X64 |`curl -L --output - https://github.com/codefresh-io/cli-v2/releases/latest/download/cf-linux-amd64.tar.gz | tar zx && mv ./cf-linux-amd64 /usr/local/bin/cf && cf version` | | ||
| | | Linux - ARM | `curl -L --output - https://github.com/codefresh-io/cli-v2/releases/latest/download/cf-linux-arm64.tar.gz | tar zx && mv ./cf-linux-arm64 /usr/local/bin/cf && cf version`| | ||
| | `Brew` | N/A| `brew tap codefresh-io/cli && brew install cf2`| | ||
| ## Failed to bootstrap repository: authentication required | ||
| ### Possible cause | ||
| The Git Runtime token provided for installation is not valid. | ||
| ### Possible actions | ||
| * Make sure the token: | ||
| * Has a valid expiration date. | ||
| * Scope includes `repo` and `admin-repo.hook` | ||
| ## Failed adding git integration <...> 404 not found | ||
| ### Possible cause | ||
| Another process is probably occupying the address you provided as your `ingress-host`. | ||
| ### Possible actions | ||
| * Terminate the process to free up the address. | ||
| ## Failed to create default git integration:...failed making a graphql API call...the HTTP request failed | ||
| ### Possible cause | ||
| The Ingress controller does not have a valid SSL certificate. The certificate must be from a Certificate Authority. | ||
| ### Possible actions | ||
| 1. Get a valid CA-signed certificate for the Ingress controller. | ||
| 1. Run the Runtime installation. | ||
| _OR_ | ||
| * Continue with the runtime installation in `insecure` mode: | ||
| `cf intg git add default --api-url https://api.github.com --runtime <runtime_name> --insecure` | ||
| where: | ||
| `<runtime_name>` is the name of the runtime installation. | ||
| ## Failed to create default git integration:...GraphQL Error (Code: 401): ("response":{"error":"Unauthorized: Invalid token."status":401, "headers":{}}..) | ||
| ### Possible cause | ||
| The ingress host specified does not point to the cluster selected for runtime installation. | ||
| ### Possible actions | ||
| Make sure the ingress host points to the cluster on which the runtime is installed. | ||
| ## Timeout error, either for applications or while waiting for the installation to complete | ||
| ### Possible cause | ||
| `Cron-executer` has identified that not all your applications are synced and healthy. Runtime installation syncs app status at 10-minute intervals. The timeout occurs if after the last status sync, at least one application is either not synced or healthy. | ||
| ### Possible actions | ||
| Without terminating the runtime install, do the following: | ||
| 1. In the Argo CD UI, check the status of your apps: | ||
| * Port forward your `argo-cd server` pod using default 8080 ports. | ||
| * Access it via `localhost:8080` with the Username `admin`, and Password which is the decoded `argocd-initial-admin-secret` in your cluster. | ||
| > With `k9s`, in the `secrets` view, press `x`. | ||
| 1. If all apps are perfectly synced and healthy, check the logs of the `cron-executor` service. | ||
| 1. If the logs don't show a cause for the error, try freeing up resources: | ||
| * Free up memory with `docker system prune -a --volumes`. | ||
| * Increase the resources allocated to your cluster. | ||
| ## Unable to communicate with `<runtime-name>` or two or more Runtimes | ||
| ### Possible cause | ||
| This error points to an issue with your browser or with your Runtime configuration, and can be due to any of the following: | ||
| {{site.data.callout.callout_tip}} | ||
| **TIP** | ||
| Verify that your internet connection is stable. | ||
| {{site.data.callout.end}} | ||
| 1. [Browser issues](#browser-issues) | ||
| Can occur because of Cross-Origin Request blocked errors. | ||
| 1. [Missing SSL certificate](#missing-ssl-certificate) | ||
| Can occur if your Runtime is ingress-based, and without an SSL certificate for the ingress controller. | ||
| 1. [Insecure protocol prefix](#insecure-protocol-prefix) | ||
| Can occur if your Runtime is ingress-based, and using HTTP for the `ingress-host` value instead of HTTPS. | ||
| ### Possible actions | ||
| Follow the steps for the actions listed below in the order in which they are listed. | ||
| <br> | ||
| #### Browser issues | ||
| Try to identify the issue that is preventing communication by accessing your browser's Developer Tools and viewing the Console or Network tabs. | ||
| For help on identifying the specific error and the corrective action to take for it, read [MDN web docs](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS/Errors){:target="\_blank"}. | ||
| <br> | ||
| #### Missing SSL certificate | ||
| If you need an SSL certificate, copy and install the root CA certificate to the trusted root certificate store on your machine.<br><br> | ||
| OR | ||
| <br><br> | ||
| Configure the browser to trust the runtime's URL and receive content: | ||
| 1. If you are not in the **GitOps Runtimes** page, from the toolbar, click the **Settings** icon. | ||
| 1. From Configuration in the sidebar, select **GitOps Runtimes**. | ||
| 1. Select the runtime, and based on your browser, do as required: | ||
| * Chrome: Click **Advanced** and then **Proceed to site**. | ||
| * Firefox: Click **Advanced** and then **Accept the risk and continue**. | ||
| * Safari: Click **Show Certificate**, and then select **Always allow content from site**. | ||
| * Edge: Click **Advanced**, and then select **Continue to site(unsafe)**. | ||
| <br> | ||
| #### Insecure protocol prefix | ||
| To continue using the HTTP protocol for your `ingress-host` instead of HTTPS, configure your browser to allow _mixed content_ for `g.codefresh.io`. | ||
| The exact steps differ based on your browser. | ||
| {{site.data.callout.callout_warning}} | ||
| **WARNING** | ||
| HTTP connections are unencrypted and less secure. | ||
| Be aware that you are using this at your own risk. | ||
| {{site.data.callout.end}} | ||
| ##### Chrome | ||
| 1. To the left of the URL, click the Lock icon, and then select **Site settings**. | ||
| 1. On the right, scroll down to **Insecure content** and then select **Allow** as the default for `g.codefresh.io`. | ||
| 1. Click **Save**. | ||
| 1. Reload the page. | ||
| <br> | ||
| ##### Firefox | ||
| 1. To the left of the URL, click the Lock icon. | ||
| 1. Expand **Connection Secure**, and then click **More Information**. | ||
| 1. In the Page Info window that appears, click the **Security** tab. | ||
| 1. Under Connection, do one of the following: | ||
| * If available, select **Enable HTTP2**. | ||
| * If not available, select **Disable Protection on This Site**, and then click **OK**. | ||
| 1. Reload the page. | ||
| <br> | ||
| ##### Safari | ||
| 1. Go to **Preferences > Advanced**. | ||
| 1. Select **Show Develop menu in menu bar**, and close the Preferences window. | ||
| 1. From the menu bar, select **Develop**, and then select **Disable Cross-Origin Restrictions**. | ||
| 1. Reload the page. | ||
| <br> | ||
| ##### Edge | ||
| 1. To the left of the URL, click the Lock icon, and then select **Permissions for this site**. | ||
| 1. Scroll down to **Location**, and then select **Allow** as the default for `g.codefresh.io`. | ||
| 1. Click **Save**. | ||
| 1. Reload the page. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.