- Notifications
You must be signed in to change notification settings - Fork8.5k
Description
What happened:
I have a pod with two services, which look like this:
apiVersion:v1kind:Servicemetadata:labels:app.kubernetes.io/instance:step-certificatesapp.kubernetes.io/managed-by:Helmapp.kubernetes.io/name:step-certificatesapp.kubernetes.io/version:0.28.4helm.sh/chart:step-certificates-1.28.4name:step-certificatesnamespace:project-finatixspec:ports: -name:httpsport:443protocol:TCPtargetPort:9000selector:app.kubernetes.io/instance:step-certificatesapp.kubernetes.io/name:step-certificatestype:ClusterIP---apiVersion:v1kind:Servicemetadata:name:step-certificates-no-sslnamespace:project-finatixspec:ports: -name:httpport:80protocol:TCPtargetPort:9001selector:app.kubernetes.io/instance:step-certificatesapp.kubernetes.io/name:step-certificatestype:ClusterIP
These services are exposed through the following ingress:
apiVersion:networking.k8s.io/v1kind:Ingressmetadata:annotations:nginx.ingress.kubernetes.io/force-ssl-redirect:"true"nginx.ingress.kubernetes.io/ssl-passthrough:"true"labels:app.kubernetes.io/instance:step-certificatesapp.kubernetes.io/managed-by:Helmapp.kubernetes.io/name:step-certificatesapp.kubernetes.io/version:0.28.4helm.sh/chart:step-certificates-1.28.4name:step-certificatesnamespace:project-finatixspec:ingressClassName:nginxrules: -host:ca.example.comhttp:paths: -backend:service:name:step-certificatesport:number:443path:/pathType:ImplementationSpecific---apiVersion:networking.k8s.io/v1kind:Ingressmetadata:annotations:nginx.ingress.kubernetes.io/force-ssl-redirect:"false"nginx.ingress.kubernetes.io/ssl-redirect:"false"name:step-certificates-ingress-no-sslnamespace:project-finatixspec:ingressClassName:nginxrules: -host:ca.example.comhttp:paths: -backend:service:name:step-certificates-no-sslport:number:80path:/1.0/crlpathType:Exact
When deploying this, both ingresses work as expected except for SSL Passthrough. It is simply not activated.
What fixes it is to remove the non-ssl ingress and immediately add it again. After that, reloading anything has no effect, it just works. This is very unexpected.
Here are some (cleaned) logs that might help.
After deploying the yaml
I1027 10:20:50.551079 7 store.go:443] "Found valid IngressClass" ingress="project-finatix/step-certificates" ingressclass="nginx"W1027 10:20:50.551878 7 controller.go:1232] Service "project-finatix/step-certificates" does not have any active Endpoint.I1027 10:20:50.552137 7 event.go:377] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"project-finatix", Name:"step-certificates", UID:"c0a205a9-fa07-4eb5-933d-ca8942cfcddd", APIVersion:"networking.k8s.io/v1", ResourceVersion:"87097835", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for syncI1027 10:20:50.582599 7 store.go:443] "Found valid IngressClass" ingress="project-finatix/step-certificates-ingress-no-ssl" ingressclass="nginx"I1027 10:20:50.583492 7 event.go:377] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"project-finatix", Name:"step-certificates-ingress-no-ssl", UID:"8d32e8fd-fd8d-49fb-ad43-2448255c1300", APIVersion:"networking.k8s.io/v1", ResourceVersion:"87097839", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for syncI1027 10:20:50.728163 7 controller.go:228] "Backend successfully reloaded"I1027 10:20:50.728726 7 event.go:377] Event(v1.ObjectReference{Kind:"Pod", Namespace:"network", Name:"ingress-nginx-controller-687dd69877-brgt5", UID:"f4e247b4-29bc-46be-b7cc-de07492cb366", APIVersion:"v1", ResourceVersion:"85916824", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configurationI1027 10:20:53.750885 7 event.go:377] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"project-finatix", Name:"step-certificates-ingress-no-ssl", UID:"8d32e8fd-fd8d-49fb-ad43-2448255c1300", APIVersion:"networking.k8s.io/v1", ResourceVersion:"87097894", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for syncI1027 10:20:53.770207 7 event.go:377] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"project-finatix", Name:"step-certificates", UID:"c0a205a9-fa07-4eb5-933d-ca8942cfcddd", APIVersion:"networking.k8s.io/v1", ResourceVersion:"87097895", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for syncW1027 10:20:53.885881 7 controller.go:1232] Service "project-finatix/step-certificates-no-ssl" does not have any active Endpoint.W1027 10:20:53.885922 7 controller.go:1232] Service "project-finatix/step-certificates" does not have any active Endpoint.I1027 10:20:53.892033 7 controller.go:214] "Configuration changes detected, backend reload required"I1027 10:20:54.044149 7 controller.go:228] "Backend successfully reloaded"I1027 10:20:54.044841 7 event.go:377] Event(v1.ObjectReference{Kind:"Pod", Namespace:"network", Name:"ingress-nginx-controller-687dd69877-brgt5", UID:"f4e247b4-29bc-46be-b7cc-de07492cb366", APIVersion:"v1", ResourceVersion:"85916824", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configurationW1027 10:20:57.220644 7 controller.go:1232] Service "project-finatix/step-certificates-no-ssl" does not have any active Endpoint.W1027 10:20:57.220687 7 controller.go:1232] Service "project-finatix/step-certificates" does not have any active Endpoint.W1027 10:21:01.097007 7 controller.go:1232] Service "project-finatix/step-certificates-no-ssl" does not have any active Endpoint.The (https!) request reaches the nginx ingress for some reason
host="ca.finatix.cloud" status=400 x_forward_for="127.0.0.1" bytes_sent=146 http_referrer="-" http_user_agent="curl/8.11.1" req_id="8d7f9e2e3e21d1e8639a8a89da7db644" req_time=0.003 req_method="GET" req_proto="HTTP/2.0" req_path="/" req_query="-" req_length=32 upstream_status=400 upstream_response_time=0.003 upstream_response_length=48 upstream_addr="10.244.2.55:9000" pstream_namespace="project-finatix" upstream_ingress="step-certificates" upstream_service="step-certificates" upstream_port=443Removing the non-ssl ingress
I1027 10:25:51.342337 7 controller.go:214] "Configuration changes detected, backend reload required"I1027 10:25:51.529354 7 controller.go:228] "Backend successfully reloaded"I1027 10:25:51.530549 7 event.go:377] Event(v1.ObjectReference{Kind:"Pod", Namespace:"network", Name:"ingress-nginx-controller-687dd69877-brgt5", UID:"f4e247b4-29bc-46be-b7cc-de07492cb366", APIVersion:"v1", ResourceVersion:"85916824", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configurationRedeploying the yaml
I1027 10:26:16.198012 7 store.go:443] "Found valid IngressClass" ingress="project-finatix/step-certificates-ingress-no-ssl" ingressclass="nginx"W1027 10:26:16.198964 7 controller.go:663] Ignoring SSL Passthrough for location "/1.0/crl" in server "ca.finatix.cloud"I1027 10:26:16.199023 7 event.go:377] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"project-finatix", Name:"step-certificates-ingress-no-ssl", UID:"0e88e750-f879-44c1-b3ab-b2ea3a816065", APIVersion:"networking.k8s.io/v1", ResourceVersion:"87099459", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for syncI1027 10:26:16.205151 7 controller.go:214] "Configuration changes detected, backend reload required"I1027 10:26:16.409708 7 controller.go:228] "Backend successfully reloaded"I1027 10:26:16.410361 7 event.go:377] Event(v1.ObjectReference{Kind:"Pod", Namespace:"network", Name:"ingress-nginx-controller-687dd69877-brgt5", UID:"f4e247b4-29bc-46be-b7cc-de07492cb366", APIVersion:"v1", ResourceVersion:"85916824", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configurationAfter these steps, it works. No logs of requests are found after that, which I believe is expected since the requests never reach nginx.
What you expected to happen: SSL Passthrough works out of the box.
NGINX Ingress controller version:
/nginx-ingress-controller --version-------------------------------------------------------------------------------NGINX Ingress controller Release: v1.13.3 Build: 93851f05e61d99eea49140c9be73499a3cb92ccc Repository: https://github.com/kubernetes/ingress-nginx nginx version: nginx/1.27.1-------------------------------------------------------------------------------Kubernetes version (usekubectl version):
kubectl version Client Version: v1.33.3Kustomize Version: v5.6.0Server Version: v1.33.0How to reproduce this issue:
Create two ingresses and two services, deploy them both in a yaml.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status