标签：通过 sts containe -o rda work 入口 alc doc
ingress做为k8s集群的入口非常重要，能实现ingress功能的软件很多，可根据自身需求选择。本篇博客主要使用nginx官方提供的nginx-ingress完成了http/https7层代理和tcp四层代理的环境配置。1，k8s的版本为1.8.2
2，docker ce的版本为19.03.8-3
3，五台主机操作系统版本为centos7，kernel版本3.10.0-957
4，使用五台主机部署，3台master节点+2台work节点
5，Ingress测试域名t.myk8s.com
nginx服务的manifest：
---
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
name: nginx
spec:
selector:
matchLabels:
app: nginx
replicas: 2 # tells deployment to run 2 pods
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.17
livenessProbe: # 设置存活探针，如果检测失败则重启pod
httpGet:
path: /
port: 80
scheme: HTTP
initialDelaySeconds: 30 # 启动容器手册监控检查的等待时间
timeoutSeconds: 5 # 健康检查请求的超时时间，如果超时，则重启该pod
readinessProbe: # 设置存活探针，通过探针检测的Pod才会像其转发请求
httpGet:
path: /
port: 80
scheme: HTTP
ports:
- containerPort: 80 # 设置为pod中应用监听的端口
---
apiVersion: v1
kind: Service
metadata:
name: nginx
namespace: default
labels:
app: nginx
spec:
ports:
- name: http
port: 80 # service服务访问的端口
protocol: TCP
targetPort: 80 # 设置为pod中应用监听的端口
selector:
app: nginxapache服务的manifests：
---
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
name: httpd
spec:
selector:
matchLabels:
app: httpd
replicas: 1 # tells deployment to run 2 pods
template:
metadata:
labels:
app: httpd
spec:
containers:
- name: httpd
image: httpd:2.4
livenessProbe: # 设置存活探针，如果检测失败则重启pod
httpGet:
path: /
port: 80
scheme: HTTP
initialDelaySeconds: 30 # 启动容器手册监控检查的等待时间
timeoutSeconds: 5 # 健康检查请求的超时时间，如果超时，则重启该pod
readinessProbe: # 设置存活探针，通过探针检测的Pod才会像其转发请求
httpGet:
path: /
port: 80
scheme: HTTP
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: httpd
namespace: default
labels:
app: httpd
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: 80
selector:
app: httpd通过manifest配置k8s，使用--record选项记录命令和版本信息方便后续的升级或者回滚等操作：
# kubectl apply -f nginx-dp.yaml --record
# kubectl apply -f httpd-dp.yaml --record查看应用状态：
# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
httpd-7c456f6cf9-gb7gk 1/1 Running 0 3m15s 10.107.199.77 work2 <none> <none>
nginx-5577575ddd-499wr 1/1 Running 0 46m 10.109.125.196 work3 <none> <none>
nginx-5577575ddd-7zwmv 1/1 Running 0 48m 10.99.1.85 work1 <none> <none>查看services状态，通过services暴露出来的固定ip可以实现对应应用的访问：
# kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
httpd ClusterIP 10.104.235.200 <none> 80/TCP 45m
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 6d5h
nginx ClusterIP 10.111.5.31 <none> 80/TCP 57mIngress控制器的部署方式使用daemon-set+hostNetwork方式，且使用nginx官方提供的nginx-ingress控制器（https://blog.51cto.com/leejia/2495558）。
生成配置Ingress的manifests：
# vim http-ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: http-ingress
spec:
rules:
- host: t.myk8s.com
http:
paths:
- path: /index.html
backend:
serviceName: nginx
servicePort: 80
- path: /
backend:
serviceName: httpd
servicePort: 80
# kubectl apply -f http-ingress.yaml查看状态：
# kubectl get ingress -o wide
NAME CLASS HOSTS ADDRESS PORTS AGE
http-ingress <none> t.myk8s.com 80 102mmaster集群中其中一台master的ip为172.18.2.175，在本地pc测试访问:
~ curl -H ‘Host: t.myk8s.com‘ 172.18.2.175
<html><body><h1>It works!</h1></body></html>
? ~ curl -H ‘Host: t.myk8s.com‘ 172.18.2.175/index.html
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>我们使用自签名证书来实现https的访问：
# openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ingress.key -out ingress.crt -subj "/CN=t.myk8s.com/O=t.myk8s.com"通过secret存储证书:
# kubectl create secret tls ingress-secret --key ingress.key --cert ingress.crt
# kubectl get secrets
NAME TYPE DATA AGE
ingress-secret kubernetes.io/tls 2 2s注意如下ingress资源要和上面的secret资源在同一个namespace下面：
# vim nginx-ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: http-ingress
spec:
tls:
- hosts:
- t.myk8s.com
secretName: ingress-secret
rules:
- host: t.myk8s.com
http:
paths:
- path: /index.html
backend:
serviceName: nginx
servicePort: 80
- path: /
backend:
serviceName: httpd
servicePort: 80使用本地pc测试访问，默认http访问也会强制转https：
? ~ curl https://t.myk8s.com -k
<html><body><h1>It works!</h1></body></html>
? ~ curl http://t.myk8s.com -I -k
HTTP/1.1 301 Moved Permanently
Server: nginx/1.17.10
Date: Thu, 21 May 2020 03:53:06 GMT
Content-Type: text/html
Content-Length: 170
Connection: keep-alive
Location: https://t.myk8s.com:443/
? ~ curl https://t.myk8s.com/index.html -k
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>可以通过配置nginx-ingress控制器实现http和https同时支持访问，config map的name以及namespace要和nginx -ingress控制器使用的相同：
# vim nginx-cf.yaml
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-config
data:
ssl-redirect: "false"
# kubectl apply -f nginx-cf.yaml -n nginx-ingress
# kubectl get configmap -n nginx-ingress
NAME DATA AGE
nginx-config 1 6d19h默认nginx-ingress控制器不提供tcp和udp代理功能，但是nginx是支持的，故也可以通过调整nginx-ingress控制器来提供tcp和udp代理服务
打开nginx官方的nginx-ingress仓库，然后修改mainifests，使nginx加载tcp和udp代理的相关配置（即在nginx-ingress.yaml的args下面添加global-configuration配置）：
# git clone https://github.com/nginxinc/kubernetes-ingress/
# cd kubernetes-ingress/deployments
# git checkout v1.7.0
# vim daemon-set/nginx-ingress.yaml
- -global-configuration=$(POD_NAMESPACE)/nginx-configuration
# kubectl apply -f daemon-set/nginx-ingress.yaml生成tcp代理相关的manifests：
# vim global-configuration.yaml
apiVersion: k8s.nginx.org/v1alpha1
kind: GlobalConfiguration
metadata:
name: nginx-configuration
namespace: nginx-ingress
spec:
listeners:
- name: nginx-tcp
port: 8000
protocol: TCP
# vim global-configuration.yaml
apiVersion: k8s.nginx.org/v1alpha1
kind: GlobalConfiguration
metadata:
name: nginx-configuration
namespace: nginx-ingress
spec:
listeners:
- name: nginx-tcp
port: 8000
protocol: TCP
[root@master1 manifests]# cat transport-server-passthrough.yaml
apiVersion: k8s.nginx.org/v1alpha1
kind: TransportServer
metadata:
name: nginx-tcp
spec:
listener:
name: nginx-tcp
protocol: TCP
upstreams:
- name: nginx-app
service: nginx
port: 80
action:
pass: nginx-app
# kubectl apply -f global-configuration.yaml
# kubectl apply -f transport-server-passthrough.yaml在本地pc测试访问
? ~ curl http://172.18.2.175:8000/index.html
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>由2台水平扩容为3台，扩容期间服务无影响：
# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
httpd-7c456f6cf9-gb7gk 1/1 Running 0 27h 10.107.199.77 work2 <none> <none>
nginx-5577575ddd-pz59n 1/1 Running 0 18s 10.99.1.91 work1 <none> <none>
nginx-5577575ddd-qsrb2 1/1 Running 0 18s 10.109.125.207 work3 <none> <none>
# kubectl scale deployment nginx --replicas 3
# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
httpd-7c456f6cf9-gb7gk 1/1 Running 0 27h 10.107.199.77 work2 <none> <none>
nginx-5577575ddd-j4nq4 1/1 Running 0 13s 10.107.199.84 work2 <none> <none>
nginx-5577575ddd-pz59n 1/1 Running 0 68s 10.99.1.91 work1 <none> <none>
nginx-5577575ddd-qsrb2 1/1 Running 0 68s 10.109.125.207 work3 <none> <none>由3台水平缩容为1台，缩容期间服务无影响：
# kubectl scale deployment nginx --replicas 1
# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
httpd-7c456f6cf9-gb7gk 1/1 Running 0 27h 10.107.199.77 work2 <none> <none>
nginx-5577575ddd-pz59n 1/1 Running 0 102s 10.99.1.91 work1 <none> <none>平滑升级，先启动一个新版本的pod，然后流量切到新pod之后，再关闭老版本的pod
# kubectl set image deployment nginx nginx=nginx:1.18 --record
# kubectl rollout status deployment nginx
Waiting for deployment "nginx" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "nginx" rollout to finish: 1 old replicas are pending termination...
deployment "nginx" successfully rolled out
# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
httpd-7c456f6cf9-gb7gk 1/1 Running 0 27h 10.107.199.77 work2 <none> <none>
nginx-648f9b4559-m2b8m 1/1 Running 0 19s 10.109.125.208 work3 <none> <none>
# kubectl get deployment nginx -o yaml|grep ‘\- image:‘
- image: nginx:1.18查看历史操作deployment的命令
# kubectl rollout history deployment nginx
deployment.apps/nginx
REVISION CHANGE-CAUSE
1 kubectl apply --filename=nginx-dp.yaml --record=true
2 kubectl set image deployment nginx nginx=nginx:1.18 --record=true可以通过两种方式回滚：
第一种：回滚最近一次操作
# kubectl rollout undo deployment nginx
# kubectl get deployment nginx -o yaml|grep ‘\- image:‘
- image: nginx:1.17第二种：回滚到指定的命令版本，即执行对应的命令
# kubectl rollout undo deployment nginx --to-revision=1https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#deploymentspec-v1-apps
https://kubernetes.github.io/ingress-nginx/user-guide/tls/
https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/
https://docs.nginx.com/nginx-ingress-controller/configuration/global-configuration/globalconfiguration-resource/
https://docs.nginx.com/nginx-ingress-controller/configuration/transportserver-resource/
k8s ingress实现http/https7层和tcp四层代理
标签：通过 sts containe -o rda work 入口 alc doc
原文地址：https://blog.51cto.com/leejia/2497454
