217, k8s 总章

1，安装 下载生成证书工具
wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
chmod +x cfssl*
mv cfssl_linux-amd64 /usr/local/bin/cfssl
mv cfssljson_linux-amd64 /usr/local/bin/cfssljson
mv cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo
2，生成证书：
执行：./etcd-cert.sh
tar xf etcd-v3.3.10-linux-amd64.tar.gz  && cd  etcd-v3.3.10-linux-amd64
cp etcd etcdctl /opt/etcd/bin
cp ca.pem  server*pem /opt/etcd/ssl/
./etcd.sh  etcd01 192.168.132.166 etcd02=https://192.168.132.167:2380,etcd03=https://192.168.132.168:2380
//黄色的是本机IP，执行脚本的这个机器IP。
systemctl start etcd  (启动前先改下各自的/opt/etcd/cfg/etcd 这个文件的IP)
完成etcd启动！
3，
scp -r  /opt/etcd/*   root@192.168.132.167:/opt/etcd/
scp -r  /opt/etcd/*   root@192.168.132.168:/opt/etcd/
 [root@localhost opt]# scp /usr/lib/systemd/system/etcd.service root@192.168.132.167:/usr/lib/systemd/system/
到这etcd 集群搭建完成！！（直接scp完，修改cfg/etcd 文件的IP 就可以直接启动）

二：
node安装docker
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum makecache fast
yum -y install docker-ce
systemctl start docker

node 部署flanneld:
写入分配的子网段到etcd，供flanneld使用。（这句在master端 操作一次就可以了）
/opt/etcd/bin/etcdctl --ca-file=/opt/etcd/ssl/ca.pem --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --endpoints="https://192.168.132.166:2379,https://192.168.132.167:2379,https://192.168.132.168:2379" set /coreos.com/network/config ‘{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}‘
//endpoints 这个为etcd集群的地址

tar xf flannel-v0.10.0-linux-amd64.tar.gz
mkdir -p /opt/kubernetes/{ssl,bin,cfg}
scp flanneld mk-docker-opts.sh  /opt/kubernetes/bin

cat <<EOF >/opt/kubernetes/cfg/flanneld

FLANNEL_OPTIONS="--etcd-endpoints=${ETCD_ENDPOINTS} \
-etcd-cafile=/opt/etcd/ssl/ca.pem -etcd-certfile=/opt/etcd/ssl/server.pem -etcd-keyfile=/opt/etcd/ssl/server-key.pem"

EOF

红色部分改为如上截图
cat <<EOF >/usr/lib/systemd/system/flanneld.service
[Unit]
Description=Flanneld overlay address etcd agent
After=network-online.target network.target
Before=docker.service

[Service]
Type=notify
EnvironmentFile=/opt/kubernetes/cfg/flanneld
ExecStart=/opt/kubernetes/bin/flanneld --ip-masq \$FLANNEL_OPTIONS
ExecStartPost=/opt/kubernetes/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/subnet.env
Restart=on-failure

[Install]
WantedBy=multi-user.target

EOF

mkdir /opt/etcd/ssl -p
从master端：
scp ca.pem  server*pem  /opt/etcd/ssl

systemctl start flanneld

在修改docker的service 如下：
EnvironmentFile=/run/flannel/subnet.env
ExecStart=/usr/bin/dockerd $DOCKER_NETWORK_OPTIONS

systemctl daemon-relaod
systemctl restart docker

三：
Master 安装APIserver：
    kubernetes-server-linux-amd64.tar.gz
    master.zip
unzip master.zip
tar xf  kubernetes-server-linux-amd64.tar.gz
[root@localhost bin]# pwd
/root/kubernetes/server/bin
mkdir -p /opt/kubernetes/{bin,ssl,cfg}
# cp kube-apiserver kubectl kube-controller-manager kube-scheduler /opt/kubernetes/bin/
创建k8s-cert 证书：
# cp ca.pem server*.pem /opt/kubernetes/ssl/
[root@localhost ~]# ./apiserver.sh 192.168.132.166 https://192.168.132.166:2379,https://192.168.132.167:2379,https://192.168.132.166:2379
[root@localhost ~]# head -c 16 /dev/urandom |od -An -t x|tr -d " "
c119666bfb3c18c8da8130a0a1c707a2
[root@localhost ~]# vim  /opt/kubernetes/cfg/token.csv
c119666bfb3c18c8da8130a0a1c707a2,kubelet-bootstrap,10001,system:kubelet-bootstrap
[root@localhost ~]#
[root@localhost ssl]# cp ca-key.pem  /opt/kubernetes/ssl/
[root@localhost ~]# systemctl start kube-apiserver


Master  controller-manager安装
[root@localhost ~]# ./controller-manager.sh 127.0.0.1

Matser  scheduler安装：
[root@localhost ~]# ./ scheduler.sh 127.0.0.1

Node 节点的安装：（必须关闭swap）
1,先拷贝node包到node节点
2,在scp  soft/kubernets/server/bin/kubelet  kube-proxy  root@nodeIP:/opt/kubernetes/bin/
2,在master生成kubeconfig
# ./kubeconfig.sh 192.168.132.176  /root/k8s-cret/   //备注：176为本机IP
scp bootstrap.kubeconfig  kube-proxy.kubeconfig  node节点的 /opt/kubernetes/cfg/
# ./kubelet.sh 192.168.132.179 10.0.0.2  //备注：192.168.132.179为你当前节点的IP   10.10.10.2为你的DNS地址
查看kubelete是否启动
发现有错误日志,创建证权限拒绝
error: failed to run Kubelet: cannot create certificate signing request: certificatesigningrequests.certificates.k8s.io is forbidden: User "kubelet-bootstrap" cannot create certificatesigningrequests.certificates.k8s.io at the cluster scope: clusterrole.rbac.authorization.k8s.io "system:node-bootstrap" not found
解决方法
在master端创建权限分配角色
[root@master ssl]# kubectl create clusterrolebinding kubelet-bootstrap --clusterrole=system:node-bootstrapper --user=kubelet-bootstrap

# ./proxy.sh 192.168.132.179   //179为当前节点的名字
[root@localhost bin]# kubectl get csr
NAME                                                   AGE   REQUESTOR           CONDITION
node-csr-BYEi6huAGgEwODnaAWr6DacXHUkZZgikJae6P9KoVu8   11m   kubelet-bootstrap   Pending
[root@localhost bin]#
[root@localhost bin]# kubectl certificate approve node-csr-BYEi6huAGgEwODnaAWr6DacXHUkZZgikJae6P9KoVu8
certificatesigningrequest.certificates.k8s.io/node-csr-BYEi6huAGgEwODnaAWr6DacXHUkZZgikJae6P9KoVu8 approved
[root@localhost bin]#
[root@localhost bin]# kubectl get node
NAME              STATUS   ROLES    AGE   VERSION
192.168.132.179   Ready    <none>   41s   v1.12.1
[root@localhost bin]#
到此单节点的master----node部署完成。

[root@localhost cfg]# rm -f kubelet.kubeconfig 
[root@localhost cfg]# pwd
/opt/kubernetes/cfg
# vim kube-proxy   //备注：修改这个文件的IP为本机node的IP
# vim kubelet  //备注：修改这个文件的IP为本机node的IP
 
[root@localhost cfg]# cd /opt/kubernetes/ssl/
[root@localhost ssl]# 
[root@localhost ssl]# ll
总用量 16
-rw-------. 1 root root 1277 10月 30 22:43 kubelet-client-2018-10-30-22-18-34.pem
-rw-------. 1 root root 1277 10月 30 22:43 kubelet-client-current.pem
-rw-r--r--. 1 root root 2197 10月 30 22:43 kubelet.crt
-rw-------. 1 root root 1675 10月 30 22:43 kubelet.key
[root@localhost ssl]# rm -f *
[root@localhost ssl]# ll
总用量 0
[root@localhost ssl]#
[root@localhost ~]# scp /usr/lib/systemd/system/{kubelet,kube-proxy}.service  root@192.168.132.178:/usr/lib/systemd/system/
systemctl start kubelet
systemctl start kube-proxy