|
| 1 | +--- |
| 2 | +title:"Troubleshooting GitOps Runtime installation" |
| 3 | +description:"Review how to resolve issues during Runtime installation" |
| 4 | +toc:true |
| 5 | +--- |
| 6 | + |
| 7 | + |
| 8 | + |
| 9 | +##Error: Job has reached the specified backoff limit |
| 10 | +Before initiating the installation, Codefresh automatically validates the`values.yaml` file to verify that the supplied values are correct. |
| 11 | + |
| 12 | +###Possible cause |
| 13 | +Validation errors in your`values.yaml` file. |
| 14 | + |
| 15 | + |
| 16 | +###Resolution |
| 17 | +1. Get more detailed information on the reason for the validation failure by running: |
| 18 | +`kubectl logs jobs/validate-values -n ${NAMESPACE}` |
| 19 | + where: |
| 20 | +`{NAMESPACE}` is the namespace of the Hybrid GitOps Runtime. |
| 21 | +1. Fix the errors. The table below describes the settings that are validated in the`values` file. |
| 22 | + |
| 23 | + |
| 24 | +| Setting| Validation| |
| 25 | +|----------------------|------------------------------------------------------------------------------------------------------------| |
| 26 | +|`userToken`| If explicitly defined, or defined as a`secretKeyRef` which exists in the current k8s context and the defined namespace.| |
| 27 | +|**Account permissions**| If the user has admin permissions for the account in which they are installing the runtime.| |
| 28 | +|**Runtime name**| If defined, and is unique to the account.| |
| 29 | +|**Access mode**| {::nomarkdown}<ul><li>For tunnel-based, the default, if`accountId` is defined, and matches the account of the`userToken` defined in the file.</li><li>For ingress-based, if the`hosts` array contains at least one entry that is a valid URL (successful HTTP GET).</li><li>If both tunnel-based and ingress-based access modes are disabled, if`runtime.ingressUrl` is defined.</li></ul>{:/}| |
| 30 | +|`gitCredentials`| When defined, includes a Git password either explicitly, or as a`secretKeyRef`, similar to`userToken`. The password or token has the required permissions in the Git provider.| |
| 31 | + |
| 32 | + |
| 33 | +##Error: failed converting helm release <runtime-name> to GitOops;...ISC repo not initialized |
| 34 | +This error displayed when you try to Configure the Runtime as Argo CD Application. |
| 35 | + |
| 36 | +###Possible cause |
| 37 | +The Shared Configuration Repository (`ISC`) has been created but is not yet initialized. |
| 38 | + |
| 39 | +###Resolution |
| 40 | +Wait a few seconds, and try again. |
| 41 | + |
| 42 | + |
| 43 | +##Runtime timeout errors |
| 44 | +One of these errors indicating a timeout: |
| 45 | +*`Error: "unable to initialise Codefresh Client", "error": "secrets \"codefresh-token\" not found"` |
| 46 | +* For tunnel-based access mode, the`codefresh-tunnel-client` pod fails after installation. |
| 47 | +* Pods failed error |
| 48 | + |
| 49 | +###Possible cause |
| 50 | +The Runtime installed successfully, but the Application Proxy (`cap-app-proxy`) pod is not running. |
| 51 | +This status persists for the duration that the resources necessary for App Proxy are being created and initialized. |
| 52 | + |
| 53 | +###Resolution |
| 54 | + |
| 55 | +#####Check Application Proxy (`cap-app-proxy`) pod status |
| 56 | +1. Verify the status of the`cap-app-proxy` pod: |
| 57 | +`kubectl get pods` |
| 58 | +1. If the pod is still**Pending**, check for resource constraints. |
| 59 | +1. Scale up your cluster if needed. |
| 60 | + |
| 61 | +####Add/update --wait flag in Install Runtime command |
| 62 | +The`--wait` flag in Install Runtime command controls how long the installation process waits for pods to become ready before timing out. |
| 63 | + |
| 64 | +* Do one of the following: |
| 65 | +* If already defined, increase the timeout beyond the default**5 minutes**. |
| 66 | +* If not defined, add the`--wait` flag to allow time for the pods to initialize. |
| 67 | + Example:`--wait 10m` |
| 68 | + |
| 69 | + |
| 70 | +##Invalid Git token error |
| 71 | + |
| 72 | +###Possible cause |
| 73 | +Your token is invalid because of missing scopes |
| 74 | +OR |
| 75 | +You are using a fine-grained token or one with custom scopes |
| 76 | + |
| 77 | +###Resolution |
| 78 | + |
| 79 | +#####Check token scopes |
| 80 | + |
| 81 | +**GitHub Git Runtime token** |
| 82 | +{: .table .table-bordered .table-hover} |
| 83 | +| Type| Required scopes| |
| 84 | +| --------| -----------------| |
| 85 | +|**Classic**|{::nomarkdown}<ul><li><codeclass="highlighter-rouge">repo</code></li><li><codeclass="highlighter-rouge">admin:repo_hook</code></li></ul>{:/}| |
| 86 | +|**Fine-grained** (personal or group-based)|{::nomarkdown}<ul><li>Repository access: <codeclass="highlighter-rouge">All repositories</code> or <codeclass="highlighter-rouge">Only select repositories</code> including all repos that Argo CD syncs from</li><li>Repository permissions:<ul><li>Administration: <codeclass="highlighter-rouge">Read and write</code></li><li>Commit statuses: <codeclass="highlighter-rouge">Read and write</code></li><li>Contents: <codeclass="highlighter-rouge">Read and write</code></li><li>Metadata: <codeclass="highlighter-rouge">Read-only</code></li><li>Pull requests: <codeclass="highlighter-rouge">Read and write</code></li><li>Webhooks: <codeclass="highlighter-rouge">Read and write</code></li></ul></li></ul>{:/}| |
| 87 | + |
| 88 | +{: .table .table-bordered .table-hover} |
| 89 | +| Type| Required scopes| |
| 90 | +| --------| -----------------| |
| 91 | +|**Classic**| {::nomarkdown}<ul><li><codeclass="highlighter-rouge">repo</code></li><li><codeclass="highlighter-rouge">admin:repo_hook</code></li></ul>{:/}| |
| 92 | +|**Fine-grained** (personal or group-based) | {::nomarkdown}<ul><li>Repository access: <codeclass="highlighter-rouge">All repositories</code> or <codeclass="highlighter-rouge">Only select repositories</code> including all repos that Argo CD syncs from |
| 93 | +</li><li>Repository permissions:<ul><li>Administration: <codeclass="highlighter-rouge">Read and write</code></li><li>Commit statuses: <codeclass="highlighter-rouge">Read and write</code></li><li>Contents: <codeclass="highlighter-rouge">Read and write</code></li><li>Metadata: <codeclass="highlighter-rouge">Read-only</code></li><li>Pull requests: <codeclass="highlighter-rouge">Read and write</code></li><li>Webhooks: <codeclass="highlighter-rouge">Read and write</code></li></ul></li</ul>{:/} | |
| 94 | + |
| 95 | + |
| 96 | + |
| 97 | +**GitHub Git user token** |
| 98 | +{: .table .table-bordered .table-hover} |
| 99 | +| Type| Required scopes| |
| 100 | +| ------------| -----------------| |
| 101 | +|**Classic**|{::nomarkdown}<ul><li><codeclass="highlighter-rouge">repo</code></li></ul>{:/}| |
| 102 | +|**Fine-grained**|{::nomarkdown}<ul><li>Repository access: <codeclass="highlighter-rouge">All repositories</code> or <codeclass="highlighter-rouge">Only select repositories</code></li><li>Repository permissions:<ul><li>Contents: <codeclass="highlighter-rouge">Read and write</code></li><li>Metadata: <codeclass="highlighter-rouge">Read-only</code></li></ul></li></ul>{:/}| |
| 103 | + |
| 104 | + |
| 105 | +{: .table .table-bordered .table-hover} |
| 106 | +| Type| Required scopes| |
| 107 | +| --------| -----------------| |
| 108 | +|**Classic**| {::nomarkdown}<ul><li><codeclass="highlighter-rouge">repo</code></li><li><codeclass="highlighter-rouge">admin:repo_hook</code></li></ul>{:/}| |
| 109 | +|**Fine-grained** (personal or group-based)| {::nomarkdown}<ul><li>Repository access: <codeclass="highlighter-rouge">All repositories</code> or <codeclass="highlighter-rouge">Only select repositories</code> including all repos that Argo CD syncs from</li><li>Repository permissions:<ul><li>Administration: <codeclass="highlighter-rouge">Read and write</code></li><li>Commit statuses: <codeclass="highlighter-rouge">Read and write</code></li><li>Contents: <codeclass="highlighter-rouge">Read and write</code></li><li>Metadata: <codeclass="highlighter-rouge">Read-only</code></li><li>Pull requests: <codeclass="highlighter-rouge">Read and write</code></li><li>Webhooks: <codeclass="highlighter-rouge">Read and write</code></li></ul></li></ul>{:/}| |
| 110 | + |
| 111 | + |
| 112 | +<!--- ##### Ensure token formatting |
| 113 | +Make sure the token is pasted without leading or trailing spaces.--> |
| 114 | + |
| 115 | +#####Skip validation for fine-grained GitHub tokens |
| 116 | +If using a custom-scope or fine-grained token and your_Runtime version is v0.18.0 or lower_, you must disable token validation to prevent failures. |
| 117 | + |
| 118 | +1. Check Runtime version: |
| 119 | +* In the Codefresh UI, on the toolbar, click the**Settings** icon. |
| 120 | +* From the sidebar, select**GitOps Runtimes**. |
| 121 | +* In the**Versionw** column, if the chart version is v0.18.0 or lower, continue with_step 2_. |
| 122 | + |
| 123 | +1. Add the`skipGitPermissionValidation` flag to your`values.yaml` file: |
| 124 | + |
| 125 | +```yaml |
| 126 | +app-proxy: |
| 127 | +config: |
| 128 | +skipGitPermissionValidation:"true" |
| 129 | +``` |
| 130 | +
|
| 131 | +## Could not connect to Argo CD error (Existing Argo only) |
| 132 | +When you try to Configure the Runtime as an Argo CD Application, the Runtime tries to connect to the Argo CD instance. |
| 133 | +
|
| 134 | +### Possible cause |
| 135 | +The Runtime cannot connect to the Argo CD instance due to one of these reasons: |
| 136 | +* The Runtime is not installed in the same namespace as Argo CD |
| 137 | +* The Runtime cannot reach required services:`argocd-server`, `argocd-repo-server`, `argocd-redis`. |
| 138 | +* Your Argo CD instance uses a non-default root path behind a reverse proxy. |
| 139 | + |
| 140 | +### Resolution |
| 141 | + |
| 142 | +Follow the steps in the order in which they are listed. |
| 143 | + |
| 144 | +##### 1. Check Runtime namespace |
| 145 | +Check the namespace where the Runtime is installed. It must match the Argo CD namespace. |
| 146 | + |
| 147 | +##### 2. Check service labels if using auto-detection |
| 148 | +1. If you have configured auto-detect for Argo CD services, check if the labels are correctly defined: |
| 149 | +* `argocd-server`: `app.kubernetes.io/component=server,app.kubernetes.io/part-of=argocd` |
| 150 | +* `argocd-repo-server`: `app.kubernetes.io/component=repo-server,app.kubernetes.io/part-of=argocd` |
| 151 | +* `argocd-redis`: `app.kubernetes.io/component=redis,app.kubernetes.io/part-of=argocd` |
| 152 | + |
| 153 | + |
| 154 | +<!--- Run this script to verify if the labels are correctly assigned:add the script --> |
| 155 | + |
| 156 | +##### 3. Manually configure service names and ports in values.yaml |
| 157 | +* If labels are not correctly defined, or if you are not using auto-detect, define service names and ports in the `values.yaml` file: |
| 158 | + |
| 159 | +```yaml |
| 160 | +global: |
| 161 | + external-argo-cd: |
| 162 | + server: |
| 163 | + svc: argocd |
| 164 | + port: 80 |
| 165 | + redis: |
| 166 | + svc: argocd-redis |
| 167 | + port: 6379 |
| 168 | + repoServer: |
| 169 | + svc: argocd-repo-server |
| 170 | + port: 8081 |
| 171 | +... |
| 172 | +``` |
| 173 | + |
| 174 | +##### 4. Verify Argo CD root path configuration |
| 175 | +* If your Argo CD instance is behind a reverse proxy and uses a non-default root path, configure the path in `values.yaml`: |
| 176 | + |
| 177 | +```yaml |
| 178 | +global: |
| 179 | + external-argo-cd: |
| 180 | + server: |
| 181 | + svc: argocd |
| 182 | + port: 80 |
| 183 | + rootpath: '/argocd' # example value if ArgoCD is behind a reverse proxy such as https://example.com/argocd/ |
| 184 | +... |
| 185 | +``` |
| 186 | + |
| 187 | + |
| 188 | +## Add Git Source failure |
| 189 | +TBD |