- Notifications
You must be signed in to change notification settings - Fork3.4k
Open
Description
This might be basically the same as#2160, or maybe it's just related. I'm not sure, but since this issue is closed I thought I should open a new one.
We're using TKGI to create and manage Kubernetes clusters. TKGI is also the IDP (OIDC) for the clusters. When I runtkgi get-credentials <cluster-name>, a~/.kube/config is created which contains the necessary information.
When the access token expires, kubectl is perfectly able to get a new one with the info from the config. But this Python lib isn't. I think the problem is that TKGI only writes the certificat and not the complete chain intoidp-certificate-authority-data.
Please let me stress again that kubectl doesnot have a problem with that!
What happened (please include outputs or screenshots):
$ pythonPython 3.12.8 (main, Dec 9 2024, 15:25:01) [GCC 8.5.0 20210514 (Red Hat 8.5.0-22)] on linuxType "help", "copyright", "credits" or "license" for more information.>>> from kubernetes import client, config>>> config.load_config()Traceback (most recent call last): File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request self._validate_conn(conn) File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn conn.connect() File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/urllib3/connection.py", line 741, in connect sock_and_verified = _ssl_wrap_socket_and_match_hostname( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/urllib3/connection.py", line 920, in _ssl_wrap_socket_and_match_hostname ssl_sock = ssl_wrap_socket( ^^^^^^^^^^^^^^^^ File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/urllib3/util/ssl_.py", line 460, in ssl_wrap_socket ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/urllib3/util/ssl_.py", line 504, in _ssl_wrap_socket_impl return ssl_context.wrap_socket(sock, server_hostname=server_hostname) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.12/ssl.py", line 455, in wrap_socket return self.sslsocket_class._create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.12/ssl.py", line 1041, in _create self.do_handshake() File "/usr/lib64/python3.12/ssl.py", line 1319, in do_handshake self._sslobj.do_handshake()ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1000)During handling of the above exception, another exception occurred:Traceback (most recent call last): File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request raise new_eurllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1000)The above exception was the direct cause of the following exception:Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/kubernetes/config/__init__.py", line 42, in load_config load_kube_config(**kwargs) File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/kubernetes/config/kube_config.py", line 826, in load_kube_config loader.load_and_set(config) File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/kubernetes/config/kube_config.py", line 589, in load_and_set self._load_authentication() File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/kubernetes/config/kube_config.py", line 288, in _load_authentication if self._load_auth_provider_token(): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/kubernetes/config/kube_config.py", line 307, in _load_auth_provider_token return self._load_oid_token(provider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/kubernetes/config/kube_config.py", line 413, in _load_oid_token self._refresh_oidc(provider) File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/kubernetes/config/kube_config.py", line 450, in _refresh_oidc response = client.request( ^^^^^^^^^^^^^^^ File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/kubernetes/client/api_client.py", line 373, in request return self.rest_client.GET(url, ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/kubernetes/client/rest.py", line 244, in GET return self.request("GET", url, ^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/kubernetes/client/rest.py", line 217, in request r = self.pool_manager.request(method, url, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/urllib3/_request_methods.py", line 135, in request return self.request_encode_url( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/urllib3/_request_methods.py", line 182, in request_encode_url return self.urlopen(method, url, **extra_kw) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/urllib3/poolmanager.py", line 443, in urlopen response = conn.urlopen(method, u.request_uri, **kw) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/urllib3/connectionpool.py", line 871, in urlopen return self.urlopen( ^^^^^^^^^^^^^ File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "/data/venv-ansible-vmware/lib64/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='tkgi-playground.it.nrw.de', port=8443): Max retries exceeded with url: /oauth/token/.well-known/openid-configuration (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1000)')))>>>What you expected to happen:
$ pythonPython 3.12.8 (main, Dec 9 2024, 15:25:01) [GCC 8.5.0 20210514 (Red Hat 8.5.0-22)] on linuxType "help", "copyright", "credits" or "license" for more information.>>> from kubernetes import client, config>>> config.load_config()>>>How to reproduce it (as minimally and precisely as possible):
Test withidp-certificate-authority-data containing only the certificat, not the complete chain.
Anything else we need to know?:
When I manually patchidp-certificate-authority-data in~/.kube/config to contain the complete certificat chain everything works. I think the problem is somewhere around here:
python/kubernetes/base/config/kube_config.py
Lines 422 to 454 in3e6cc58
| def_refresh_oidc(self,provider): | |
| config=Configuration() | |
| if'idp-certificate-authority-data'inprovider['config']: | |
| ca_cert=tempfile.NamedTemporaryFile(delete=True) | |
| ifPY3: | |
| cert=base64.b64decode( | |
| provider['config']['idp-certificate-authority-data'] | |
| ).decode('utf-8') | |
| else: | |
| cert=base64.b64decode( | |
| provider['config']['idp-certificate-authority-data']+"==" | |
| ) | |
| withopen(ca_cert.name,'w')asfh: | |
| fh.write(cert) | |
| config.ssl_ca_cert=ca_cert.name | |
| elif'idp-certificate-authority'inprovider['config']: | |
| config.ssl_ca_cert=provider['config']['idp-certificate-authority'] | |
| else: | |
| config.verify_ssl=False | |
| client=ApiClient(configuration=config) | |
| response=client.request( | |
| method="GET", | |
| url="%s/.well-known/openid-configuration" | |
| %provider['config']['idp-issuer-url'] | |
| ) |
When connecting to TKGI, there's the complete certificat chain. Neither kubectl nor curl nor anything else has a problem there. But this lib has, because it trusts only the certificat fromidp-certificate-authority-data, which lacks the intermediate and root CA certificats.
Environment:
- Kubernetes version (
kubectl version):
Client Version: v1.30.7Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3Server Version: v1.31.5+vmware.1- OS (e.g., MacOS 10.13.6): Red Hat Enterprise Linux release 8.10 (Ootpa)
- Python version (
python --version): 3.12.8 - Python client version (
pip list | grep kubernetes): 32.0.1