Kubernetes Ingress

Kubernetes Service基础一节中,我们了解到可以通过设置Service NodePort向Kubernetes集群外部暴露端口,外部服务可以通过Kubernetes集群节点IP+NodePort访问集群内部资源。当集群内部服务众多时,需要暴露的端口也会越来越多。这样不仅端口维护困难,集群边界也变得“千疮百孔”。针对这个问题,Kubernetes提供了Ingress来解决,Ingress对象用于配置外部请求转发到集群内部服务的具体规则,而实际的转发操作由Ingress Controller来完成。

假设我们的Kubernetes集群中分别存在2个实例的tomcat和nginx Deployment,并且有对应的Service。加入Ingress后,我们可以实现如下图所示的服务暴露方式:

QQ截图20191128181259.png

Ingress Controller

Ingress Controller并不是Kubernetes对象,而是根据Ingress对象配置,实现具体转发功能的组件统称。除了Kubernetes官方维护的GCE和Ingress Nginx外,还有许多第三方维护的实现。这里以用的较多的Ingress Nginx为例,实现Ingress Controller的部署。

因为Ingress Controller是用于处理集群外部请求访问集群内部服务的组件,所以我们需要思考,如何将Ingress Controller暴露出去。最为常见的方式主要有以下两种:

  1. 创建和Intress Controller对应的Service服务,Service通过NodePort将服务端口暴露出去;

  2. 将Ingress Controller部署到几个固定的节点上,然后通过HostPort将端口映射出去,最外层通过LVS+keepalive实现负载均衡。

因为第1种方式需要在请求链路中再加一层Service服务,性能可能会有耗损,所以我们选择第2种方式。

为了简化过程,这里只在一个节点上部署Ingress Controller,比如我们可以选择在Node1节点上部署。给Node1节点打个标签:

1
kubectl label node node1 type="ingress"

下载Ingress Nginx配置文件:

1
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml

修改该配置文件:

1
vi mandatory.yaml

修改部分如下所示:

QQ截图20191128184352.png

创建该配置文件:

1
kubectl create -f mandatory.yaml

QQ截图20191128184513.png

查看是否创建成功:

QQ截图20191128184723.png

使用浏览器访问http://192.168.33.12/

QQ截图20191128190016.png

因为还没有创建Ingress,所以页面响应暂时为404。

Ingress

在创建Ingress对象前,我们需要准备好tomcat和nginx服务,供待会演示,创建demo.yml:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: tomcat-app
spec:
selector:
matchLabels:
name: tomcat
replicas: 2
template:
metadata:
labels:
name: tomcat
spec:
containers:
- name: tomcat
image: tomcat:8.0.51-alpine
ports:
- containerPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-app
spec:
selector:
matchLabels:
name: nginx
replicas: 2
template:
metadata:
labels:
name: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: tomcat-service
spec:
ports:
- port: 8080
targetPort: 8080
selector:
name: tomcat
---
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
ports:
- port: 8081
targetPort: 80
selector:
name: nginx

创建:

1
kubectl create -f demo.yml

QQ截图20191128191135.png

查看是否创建成功:

QQ截图20191128191253.png

接着创建Ingress配置文件(ingress.yml):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress
spec:
rules:
- host: tomcat.mrbird.cc
http:
paths:
- path: /
backend:
serviceName: tomcat-service
servicePort: 8080
- host: nginx.mrbird.cc
http:
paths:
- path: /
backend:
serviceName: nginx-service
servicePort: 8081

根据上述配置,当我们访问tomcat.mrbird.cc根路径的时候,请求将转发到名称为tomcat-service,端口为8080的service上,根据上面demo.yml的配置,该service对应两个tomcat pod;访问nginx.mrbird.cc根路径的时候,请求将转发到nginx-service。

创建该Ingress:

1
kubectl create -f ingress.yml

QQ截图20191129094819.png

从图中可以看出,当我们访问tomcat.mrbird.cc/的时候,请求会均衡地转发到10.244.1.10:8080/和10.244.2.15:8080/。

在Windows上配置hosts域名解析:

QQ截图20191129095034.png

浏览器访问http://tomcat.mrbird.cc/

QQ截图20191129095222.png

访问http://nginx.mrbird.cc/

QQ截图20191129095333.png

结果符合我们的预期。

Ingress Nginx实质上就是一个nginx服务,它可以自动通过我们的Ingress配置,生成相应的nginx配置文件,我们可以进入到Ingress Nginx容器内部证实这一点:

QQ截图20191129100835.png

在配置文件中,可以看到下面这些配置(仅截取tomcat.mrbird.cc配置):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
...
## start server tomcat.mrbird.cc
server {
server_name tomcat.mrbird.cc ;

listen 80 ;
listen [::]:80 ;
listen 443 ssl http2 ;
listen [::]:443 ssl http2 ;

set $proxy_upstream_name "-";

ssl_certificate_by_lua_block {
certificate.call()
}

location / {

set $namespace "default";
set $ingress_name "ingress";
set $service_name "tomcat-service";
set $service_port "8080";
set $location_path "/";

rewrite_by_lua_block {
lua_ingress.rewrite({
force_ssl_redirect = false,
ssl_redirect = true,
force_no_ssl_redirect = false,
use_port_in_redirects = false,
})
balancer.rewrite()
plugins.run()
}

header_filter_by_lua_block {

plugins.run()
}
body_filter_by_lua_block {

}

log_by_lua_block {

balancer.log()

monitor.call()

plugins.run()
}

port_in_redirect off;

set $balancer_ewma_score -1;
set $proxy_upstream_name "default-tomcat-service-8080";
set $proxy_host $proxy_upstream_name;
set $pass_access_scheme $scheme;
set $pass_server_port $server_port;
set $best_http_host $http_host;
set $pass_port $pass_server_port;

set $proxy_alternative_upstream_name "";

client_max_body_size 1m;

proxy_set_header Host $best_http_host;

# Pass the extracted client certificate to the backend

# Allow websocket connections
proxy_set_header Upgrade $http_upgrade;

proxy_set_header Connection $connection_upgrade;

proxy_set_header X-Request-ID $req_id;
proxy_set_header X-Real-IP $remote_addr;

proxy_set_header X-Forwarded-For $remote_addr;

proxy_set_header X-Forwarded-Host $best_http_host;
proxy_set_header X-Forwarded-Port $pass_port;
proxy_set_header X-Forwarded-Proto $pass_access_scheme;

proxy_set_header X-Scheme $pass_access_scheme;

# Pass the original X-Forwarded-For
proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;

# mitigate HTTPoxy Vulnerability
# https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
proxy_set_header Proxy "";

# Custom headers to proxied server

proxy_connect_timeout 5s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;

proxy_buffering off;
proxy_buffer_size 4k;
proxy_buffers 4 4k;

proxy_max_temp_file_size 1024m;

proxy_request_buffering on;
proxy_http_version 1.1;

proxy_cookie_domain off;
proxy_cookie_path off;

# In case of errors try the next upstream server before returning an error
proxy_next_upstream error timeout;
proxy_next_upstream_timeout 0;
proxy_next_upstream_tries 3;

proxy_pass http://upstream_balancer;

proxy_redirect off;

}

}
## end server tomcat.mrbird.cc
...

更多Ingress配置可以参考官方文档:https://v1-12.docs.kubernetes.io/zh/docs/concepts/services-networking/ingress/

请作者喝瓶肥宅水~

TOP