码迷,mamicode.com
首页 > Web开发 > 详细

Kubernetes集群搭建以及部署高可用ingress

时间:2020-03-18 11:18:55      阅读:75      评论:0      收藏:0      [点我收藏+]

标签:mic   erb   sse   ingress   cto   工具安装   ISE   部署   his   

k8s服务器

10.10.4.85 master

10.10.4.86 node

10.10.4.87 node

3个服务器都配置好主机名以及hosts配置:

技术图片

k8s集群搭建

在10.10.4.85 上操作,master节点

使用kubeadm工具安装,参考:使用kubeadm安装kubernetes_v1.17.x:https://kuboard.cn/install/install-k8s.htm

#curl -sSL https://kuboard.cn/install-script/v1.17.x/init_master.sh | sh -s 1.17.3
#kubectl get pod -n kube-system -o wide
检查pod启动情况,若报错,请检查具体pod:
#kubectl describe pod -n kube-system calico-node-v6zdg
若镜像报错,则手动下载镜像
比如:
#docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.1
当都启动好后,用save命令将镜像都打包,然后传给各个node
#docker save -o k8s.tar k8s的镜像    
#scp k8s.tar 10.10.4.86
#scp k8s.tar 10.10.4.87
获取token:
#kubeadm token create --print-join-command
?

在10.10.4.86 和10.10.4.87 上操作,是node节点

使用kubeadm工具安装,参考: 初始化worker节点:https://kuboard.cn/install/install-k8s.html#初始化-worker节点

导入镜像
#docker load < k8s.tar
加入集群
#kubeadm join apiserver.vonedao:6443 --token y4gc5t.2eafib5oebrl3rm6     --discovery-token-ca-cert-hash sha256:481641cfbb04c5330d3cf90377589d2e657153a5ec23cbe416131f30f60268ee
?

NFS搭建

nfs用于做数据存储,比如jenkins的

安装软件(全部机器安装):

#yum install -y nfs-utils rpcbind

10.10.4.85 配置:

#vim /etc/exports
/data *(rw,sync,no_root_squash)

10.10.4.85 配置:

#vi /etc/sysconfig/nfs
RPCNFSDARGS=""
RPCMOUNTDOPTS=""
STATDARG=""
SMNOTIFYARGS=""
RPCIDMAPDARGS=""
RPCGSSDARGS=""
GSS_USE_PROXY="yes"
BLKMAPDARGS=""
RQUOTAD_PORT=4001
LOCKD_TCPPORT=4002
LOCKD_UDPPORT=4002
MOUNTD_PORT=4003
STATD_PORT=4004

10.10.4.85 重启、开机自启动:

#systemctl restart rpcbind && systemctl restart nfs
#systemctl enable rpcbind
#systemctl enable nfs

节点机器查看:

#showmount -e 10.10.4.85

部署高可用Ingress

技术图片

由多个独占Ingress实例组成统一接入层承载集群入口流量,同时可依据后端业务流量水平扩缩容Ingress节点。当然如果您前期的集群规模并不大,也可以采用将Ingress服务与业务应用混部的方式,但建议进行资源限制和隔离。

想法是美好的,服务器少了,无SLB,无公网,内网搭建多ingress练习。

ingress的高可用的话,要可以通过把nginx-ingress-controller运行到指定添加标签的几个node节点上,然后再把这几个node节点加入到LB中,然后对应的域名解析到该LB即可实现ingress的高可用。(注意:添加标签的节点数量要大于等于集群Pod副本数,从而避免多个Pod运行在同一个节点上。不建议将Ingress服务部署到master节点上,尽量选择worker节点添加标签。

注意,以下操作都在10.10.4.85操作。 文件目录:/opt/ ,基础的yaml文件都放这里。

技术图片

 

 

给node添加标签

节点都部署ingress,用于做冗余。首先在master上给节点打标签:

技术图片

 

 # kubectl label node vonedaonode1 ingresscontroller=true

# kubectl label node vonedaonode3 ingresscontroller=true

查看标签
#kubectl get nodes --show-labels

技术图片

下载mandatory.yaml

/opt/ingress

从官网下载:mandatory.yaml

https://github.com/kubernetes?page=2

技术图片

技术图片

技术图片

打开后,在浏览器页面复制或者wget这个地址获取文件。

修改mandatory.yaml

apiVersion: v1
kind: Namespace
metadata:
name: ingress-nginx
labels:
  app.kubernetes.io/name: ingress-nginx
  app.kubernetes.io/part-of: ingress-nginx
?
---
?
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-configuration
namespace: ingress-nginx
labels:
  app.kubernetes.io/name: ingress-nginx
  app.kubernetes.io/part-of: ingress-nginx
?
---
kind: ConfigMap
apiVersion: v1
metadata:
name: tcp-services
namespace: ingress-nginx
labels:
  app.kubernetes.io/name: ingress-nginx
  app.kubernetes.io/part-of: ingress-nginx
?
---
kind: ConfigMap
apiVersion: v1
metadata:
name: udp-services
namespace: ingress-nginx
labels:
  app.kubernetes.io/name: ingress-nginx
  app.kubernetes.io/part-of: ingress-nginx
?
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
labels:
  app.kubernetes.io/name: ingress-nginx
  app.kubernetes.io/part-of: ingress-nginx
?
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: nginx-ingress-clusterrole
labels:
  app.kubernetes.io/name: ingress-nginx
  app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
    - ""
  resources:
    - configmaps
    - endpoints
    - nodes
    - pods
    - secrets
  verbs:
    - list
    - watch
- apiGroups:
    - ""
  resources:
    - nodes
  verbs:
    - get
- apiGroups:
    - ""
  resources:
    - services
  verbs:
    - get
    - list
    - watch
- apiGroups:
    - ""
  resources:
    - events
  verbs:
    - create
    - patch
- apiGroups:
    - "extensions"
    - "networking.k8s.io"
  resources:
    - ingresses
  verbs:
    - get
    - list
    - watch
- apiGroups:
    - "extensions"
    - "networking.k8s.io"
  resources:
    - ingresses/status
  verbs:
    - update
?
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
name: nginx-ingress-role
namespace: ingress-nginx
labels:
  app.kubernetes.io/name: ingress-nginx
  app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
    - ""
  resources:
    - configmaps
    - pods
    - secrets
    - namespaces
  verbs:
    - get
- apiGroups:
    - ""
  resources:
    - configmaps
  resourceNames:
     # Defaults to "<election-id>-<ingress-class>"
     # Here: "<ingress-controller-leader>-<nginx>"
     # This has to be adapted if you change either parameter
     # when launching the nginx-ingress-controller.
    - "ingress-controller-leader-nginx"
  verbs:
    - get
    - update
- apiGroups:
    - ""
  resources:
    - configmaps
  verbs:
    - create
- apiGroups:
    - ""
  resources:
    - endpoints
  verbs:
    - get
?
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: nginx-ingress-role-nisa-binding
namespace: ingress-nginx
labels:
  app.kubernetes.io/name: ingress-nginx
  app.kubernetes.io/part-of: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: nginx-ingress-role
subjects:
- kind: ServiceAccount
  name: nginx-ingress-serviceaccount
  namespace: ingress-nginx
?
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: nginx-ingress-clusterrole-nisa-binding
labels:
  app.kubernetes.io/name: ingress-nginx
  app.kubernetes.io/part-of: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: nginx-ingress-clusterrole
subjects:
- kind: ServiceAccount
  name: nginx-ingress-serviceaccount
  namespace: ingress-nginx
?
---
?
apiVersion: apps/v1
#kind: Deployment 换成DaemonSet
kind: DaemonSet    
metadata:
name: nginx-ingress-controller
namespace: ingress-nginx
labels:
  app.kubernetes.io/name: ingress-nginx
  app.kubernetes.io/part-of: ingress-nginx
spec:
 #replicas: 1   注释掉
selector:
  matchLabels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
template:
  metadata:
    labels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
    annotations:
      prometheus.io/port: "10254"
      prometheus.io/scrape: "true"
  spec:
     # wait up to five minutes for the drain of connections
    terminationGracePeriodSeconds: 300
    serviceAccountName: nginx-ingress-serviceaccount
     #启用hostNetwork网络,指定之前添加ingresscontroller=true标签的node
    hostNetwork: true
    nodeSelector:
      ingresscontroller: ‘true‘
    containers:
      - name: nginx-ingress-controller
        image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0
        args:
          - /nginx-ingress-controller
          - --configmap=$(POD_NAMESPACE)/nginx-configuration
          - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
          - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
          - --publish-service=$(POD_NAMESPACE)/ingress-nginx
          - --annotations-prefix=nginx.ingress.kubernetes.io
        securityContext:
          allowPrivilegeEscalation: true
          capabilities:
            drop:
              - ALL
            add:
              - NET_BIND_SERVICE
           # www-data -> 101
          runAsUser: 101
        env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
        ports:
          - name: http
            containerPort: 80
            protocol: TCP
          - name: https
            containerPort: 443
            protocol: TCP
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 10
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 10
        lifecycle:
          preStop:
            exec:
              command:
                - /wait-shutdown
?
---
?
apiVersion: v1
kind: LimitRange
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
  app.kubernetes.io/name: ingress-nginx
  app.kubernetes.io/part-of: ingress-nginx
spec:
limits:
- min:
    memory: 90Mi
    cpu: 100m
  type: Container

部署ingress

#kubectl create -f mandatory.yaml

#kubectl get pods -n ingress-nginx -o wide

技术图片

 

Kubernetes集群搭建以及部署高可用ingress

标签:mic   erb   sse   ingress   cto   工具安装   ISE   部署   his   

原文地址:https://www.cnblogs.com/zoujiaojiao/p/12515917.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!