我有太多的LoadBalancer服务占用了太多的外部IP，我想切换到使用Ingress控制器。
我做了tutorial，并且在Google提供的广告连播中一切正常。
但是，通过我的pod，我可以使用NodePort服务...
? >curl http://35.223.89.81:32607/healthz
OK ? >
...但是对Ingress Controller的调用始终失败...
? >curl http://35.241.21.71:80/healthz
<!DOCTYPE html>
<html lang=en>
<meta charset=utf-8>
<meta name=viewport content="initial-scale=1,minimum-scale=1,width=device-width">
<title>Error 404 (Not Found)!!1</title>
<style>
*{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}
</style>
<a href=//www.google.com/><span id=logo aria-label=Google></span></a>
<p><b>404.</b> <ins>That’s an error.</ins>
<p>The requested URL <code>/healthz</code> was not found on this server. <ins>That’s all we know.</ins>
这是我使用的k8s版本：
? >gcloud container clusters list
NAME LOCATION MASTER_VERSION MASTER_IP MACHINE_TYPE NODE_VERSION NUM_NODES STATUS
monza-predictors us-central1-a 1.13.11-gke.14 35.193.247.210 n1-standard-1 1.13.11-gke.9 * 2 RUNNING
入口的YAML
? >cat fanout-ingress-v2.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: fanout-ingress
spec:
rules:
- http:
paths:
- path: /healthz
backend:
serviceName: predictor-classification-seatbelt-driver-service-node-port
servicePort: 4444
- path: /seatbelt-driver
backend:
serviceName: predictor-classification-seatbelt-driver-service-node-port
servicePort: 4444
描述入口
? >kubectl describe ing fanout-ingress
Name: fanout-ingress
Namespace: default
Address: 35.241.21.71
Default backend: default-http-backend:80 (10.40.2.10:8080)
Rules:
Host Path Backends
---- ---- --------
*
/healthz predictor-classification-seatbelt-driver-service-node-port:4444 (<none>)
/seatbelt-driver predictor-classification-seatbelt-driver-service-node-port:4444 (<none>)
Annotations:
ingress.kubernetes.io/url-map: k8s-um-default-fanout-ingress--62f4c45447b62142
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{},"name":"fanout-ingress","namespace":"default"},"spec":{"rules":[{"http":{"paths":[{"backend":{"serviceName":"predictor-classification-seatbelt-driver-service-node-port","servicePort":4444},"path":"/healthz"},{"backend":{"serviceName":"predictor-classification-seatbelt-driver-service-node-port","path":"/seatbelt-driver"}]}}]}}
ingress.kubernetes.io/backends: {"k8s-be-31413--62f4c45447b62142":"HEALTHY","k8s-be-32607--62f4c45447b62142":"UNHEALTHY"}
ingress.kubernetes.io/forwarding-rule: k8s-fw-default-fanout-ingress--62f4c45447b62142
ingress.kubernetes.io/target-proxy: k8s-tp-default-fanout-ingress--62f4c45447b62142
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ADD 21m loadbalancer-controller default/fanout-ingress
Normal CREATE 19m loadbalancer-controller ip: 35.241.21.71
我注意到2个后端中的1个不健康。
NodePort服务的YAML：
? >cat service-node-port-classification-predictor.yaml
apiVersion: v1
kind: Service
metadata:
name: predictor-classification-seatbelt-driver-service-node-port
namespace: default
spec:
ports:
- port: 4444
protocol: TCP
targetPort: 4444
selector:
app: predictor-classification-seatbelt-driver
type: NodePort
描述NodePort服务
? >kubectl describe svc predictor-classification-seatbelt-driver-service-node-port
Name: predictor-classification-seatbelt-driver-service-node-port
Namespace: default
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"v1","kind":"Service","name":"predictor-classification-seatbelt-driver-service-node-port","name...
Selector: app=predictor-classification-seatbelt-driver
Type: NodePort
IP: 10.43.243.69
Port: <unset> 4444/TCP
TargetPort: 4444/TCP
NodePort: <unset> 32607/TCP
Endpoints: 10.40.2.16:4444
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
用于部署的YAML
? >cat deployment-classification-predictor-v2.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: predictor-classification-seatbelt-driver
labels:
app: predictor-classification-seatbelt-driver
spec:
replicas: 1
selector:
matchLabels:
app: predictor-classification-seatbelt-driver
template:
metadata:
labels:
app: predictor-classification-seatbelt-driver
spec:
containers:
- name: predictor-classification-seatbelt-driver
image: gcr.io/annotator-1286/classification-predictor
command: ["/app/server.sh"]
args: ["4444","https://storage.googleapis.com/com-aosvapps-runs/38/1564677191/models/mobile.pb","https://storage.googleapis.com/com-aosvapps-runs/38/1564677191/models/labels.csv"]
ports:
- containerPort: 4444
livenessProbe:
httpGet:
path: /healthz
port: 4444
initialDelaySeconds: 120
描述部署
? >kubectl describe deploy predictor-classification-seatbelt-driver
Name: predictor-classification-seatbelt-driver
Namespace: default
CreationTimestamp: Mon,18 Nov 2019 12:17:13 -0800
Labels: app=predictor-classification-seatbelt-driver
Annotations: deployment.kubernetes.io/revision: 1
kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"apps/v1","kind":"Deployment","labels":{"app":"predictor-classification-seatbelt-driver"},"name...
Selector: app=predictor-classification-seatbelt-driver
Replicas: 1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable,25% max surge
pod Template:
Labels: app=predictor-classification-seatbelt-driver
Containers:
predictor-classification-seatbelt-driver:
Image: gcr.io/annotator-1286/classification-predictor
Port: 4444/TCP
Host Port: 0/TCP
Command:
/app/server.sh
Args:
4444
https://storage.googleapis.com/com-aosvapps-runs/38/1564677191/models/mobile.pb
https://storage.googleapis.com/com-aosvapps-runs/38/1564677191/models/labels.csv
Liveness: http-get http://:4444/healthz delay=120s timeout=1s period=10s #success=1 #failure=3
Environment: <none>
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
Progressing True NewReplicaSetavailable
OldReplicaSets: <none>
NewReplicaSet: predictor-classification-seatbelt-driver-85bc679444 (1/1 replicas created)
Events: <none>
豆荚的描述
? >kubectl describe po predictor-classification-seatbelt-driver-85bc679444-lcb7v
Name: predictor-classification-seatbelt-driver-85bc679444-lcb7v
Namespace: default
Priority: 0
PriorityClassname: <none>
Node: gke-monza-predictors-default-pool-268f57e3-1bs6/10.128.0.65
Start Time: Mon,18 Nov 2019 12:17:13 -0800
Labels: app=predictor-classification-seatbelt-driver
pod-template-hash=85bc679444
Annotations: kubernetes.io/limit-ranger: LimitRanger plugin set: cpu request for container predictor-classification-seatbelt-driver
Status: Running
IP: 10.40.2.16
Controlled By: ReplicaSet/predictor-classification-seatbelt-driver-85bc679444
Containers:
predictor-classification-seatbelt-driver:
Container ID: docker://90ce1466b852760db92bc66698295a2ae2963f19d26111e5be03d588dc83a712
Image: gcr.io/annotator-1286/classification-predictor
Image ID: docker-pullable://gcr.io/annotator-1286/classification-predictor@sha256:63690593d710182110e51fbd620d6944241c36dd79bce7b08b2823677ec7b929
Port: 4444/TCP
Host Port: 0/TCP
Command:
/app/server.sh
Args:
4444
https://storage.googleapis.com/com-aosvapps-runs/38/1564677191/models/mobile.pb
https://storage.googleapis.com/com-aosvapps-runs/38/1564677191/models/labels.csv
State: Running
Started: Mon,18 Nov 2019 12:17:15 -0800
Ready: True
Restart Count: 0
Requests:
cpu: 100m
Liveness: http-get http://:4444/healthz delay=120s timeout=1s period=10s #success=1 #failure=3
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-8q95m (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
podScheduled True
Volumes:
default-token-8q95m:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-8q95m
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events: <none>
更新：使用Single Service Ingress不能解决问题
? >cat fanout-ingress-v3.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: fanout-ingress
spec:
backend:
serviceName: predictor-classification-seatbelt-driver-service-node-port
servicePort: 4444
? >kubectl apply -f fanout-ingress-v3.yaml
ingress.extensions/fanout-ingress created
? >kubectl describe ing fanout-ingress
Name: fanout-ingress
Namespace: default
Address: 35.244.250.224
Default backend: predictor-classification-seatbelt-driver-service-node-port:4444 (10.40.2.16:4444)
Rules:
Host Path Backends
---- ---- --------
* * predictor-classification-seatbelt-driver-service-node-port:4444 (10.40.2.16:4444)
Annotations:
ingress.kubernetes.io/url-map: k8s-um-default-fanout-ingress--62f4c45447b62142
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","spec":{"backend":{"serviceName":"predictor-classification-seatbelt-driver-service-node-port","servicePort":4444}}}
ingress.kubernetes.io/backends: {"k8s-be-32607--62f4c45447b62142":"Unknown"}
ingress.kubernetes.io/forwarding-rule: k8s-fw-default-fanout-ingress--62f4c45447b62142
ingress.kubernetes.io/target-proxy: k8s-tp-default-fanout-ingress--62f4c45447b62142
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ADD 3m31s loadbalancer-controller default/fanout-ingress
Normal CREATE 2m56s loadbalancer-controller ip: 35.244.250.224
? >curl 35.244.250.224/healthz
<!DOCTYPE html>
<html lang=en>
<meta charset=utf-8>
<meta name=viewport content="initial-scale=1,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}
</style>
<a href=//www.google.com/><span id=logo aria-label=Google></span></a>
<p><b>404.</b> <ins>That’s an error.</ins>
<p>The requested URL <code>/healthz</code> was not found on this server. <ins>That’s all we know.</ins>