码迷，mamicode.com
首页 > 其他好文 > 详细

九、docker的跨主机通讯之overlay（vxlan）

时间：2020-12-17 12:16:32      阅读：3      评论：0      收藏：0      [点我收藏+]

标签：messages   hit   display   spl   oop   端口   unix   防火   类型   

在做这个实验前，需要用到consul，用于实现服务发现和DNS的功能，consul的安装相对简单，其详情功能介绍可查看官网：https://www.cnblogs.com/xiaohanlin/p/8016803.html，https://www.consul.io/docs/architecture

技术图片

 

docker run -it -p 8500:8500 -h consul --name consule --restart=always progrium/consul -server --bootstrap　　#启动consul容器，并配置为server端
[root@docker-1-26 ~]# cat /etc/docker/daemon.json
{
  "hosts":["tcp://0.0.0.0:2376","unix:///var/run/docker.sock"],
  "cluster-store":"consul://10.0.0.26:8500",　　　　#向console进行注册
  "cluster-advertise":"10.0.0.26:2376"
}
[root@docker-2-27 ~]# cat /etc/docker/daemon.json
{
  "hosts":["tcp://0.0.0.0:2376","unix:///var/run/docker.sock"],
  "cluster-store":"consul://10.0.0.26:8500",
  "cluster-advertise":"10.0.0.27:2376"
}

环境搭建完成后，可登陆consul界面查看到共三个node，到这一步，测试要用的环境就准备好了

技术图片

创建overlay类型的docker网络

[root@docker-1-26 ~]# docker network create -d overlay --subnet 20.0.0.0/24 --gateway 20.0.0.254 overlay_20201209
6dbac2cd67bc3cfcffb67f607a5719355570332ca45a8ab223710a1372ce6de2
[root@docker-1-26 ~]# docker network ls 
NETWORK ID          NAME                DRIVER              SCOPE
5ca0e6f8c027        bridge              bridge              local
2bb25566094c        host                host                local
640d652cd920        macvlan_20201206    macvlan             local
66221610ad40        none                null                local
6dbac2cd67bc        overlay_20201209    overlay             global　　#因为搭建了集群，创建的网络类型为overlay，scope为global，那集群内的其他节点都应该会有相应的网络
[root@docker-3-28 ~]# docker network ls 
NETWORK ID          NAME                DRIVER              SCOPE
0a4efb4f841d        bridge              bridge              local
2bb25566094c        host                host                local
a22940610925        macvlan_20201206    macvlan             local
66221610ad40        none                null                local
6dbac2cd67bc        overlay_20201209    overlay             global

现在三个节点分别创建容器：

[root@docker-2-27 ~]# docker run -it --network overlay_20201209 --name busybox_2 busybox:latest sh
[root@docker-3-28 ~]# docker run -it --network overlay_20201209 --name busybox_3 busybox:latest sh
/ # ip add 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
6: eth0@if7: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue 
    link/ether 02:42:14:00:00:03 brd ff:ff:ff:ff:ff:ff
    inet 20.0.0.3/24 brd 20.0.0.255 scope global eth0
       valid_lft forever preferred_lft forever
9: eth1@if10: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue 
    link/ether 02:42:ac:12:00:02 brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.2/16 brd 172.18.255.255 scope global eth1
       valid_lft forever preferred_lft forever
/ #

这个时候容器间跨容器通讯应该是正常的，实验过程中发现无法通讯，查看messages发现报错：

Dec 10 04:56:02 docker-3-28 dockerd: time="2020-12-10T04:56:02.630234941+08:00" level=error msg="2020/12/10 04:56:02 [ERR] memberlist: Push/Pull with docker-2-27 failed: dial tcp 10.0.0.27:7946: connect: no route to host\n"　　
#连接成员的7946端口有失败，检查端口状态正常
[root@docker-3-28 ~]# netstat -tnlup | grep 7946
tcp        0      0 10.0.0.28:7946          0.0.0.0:*               LISTEN      2153/dockerd        
udp        0      0 10.0.0.28:7946          0.0.0.0:*                           2153/dockerd

[root@docker-3-28 ~]# systemctl status docker
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2020-12-10 04:19:56 CST; 6min ago　　#防火墙状态开启的，关闭后容器可实现跨集群通讯

Dec 9 21:01:01 docker-1-26 systemd: Started Session 2 of user root.
Dec 9 21:01:01 docker-1-26 systemd: Starting Session 2 of user root.

/ # ping -c 4 20.0.0.3　　　　#实现了容器跨节点通讯
PING 20.0.0.3 (20.0.0.3): 56 data bytes
64 bytes from 20.0.0.3: seq=0 ttl=64 time=0.941 ms
64 bytes from 20.0.0.3: seq=1 ttl=64 time=1.355 ms
64 bytes from 20.0.0.3: seq=2 ttl=64 time=0.587 ms
64 bytes from 20.0.0.3: seq=3 ttl=64 time=0.644 ms

--- 20.0.0.3 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 0.587/0.881/1.355 ms
/ # ping -c 4 20.0.0.2
PING 20.0.0.2 (20.0.0.2): 56 data bytes
64 bytes from 20.0.0.2: seq=0 ttl=64 time=2.023 ms
64 bytes from 20.0.0.2: seq=1 ttl=64 time=0.674 ms
64 bytes from 20.0.0.2: seq=2 ttl=64 time=0.744 ms
64 bytes from 20.0.0.2: seq=3 ttl=64 time=0.616 ms

--- 20.0.0.2 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 0.616/1.014/2.023 ms
/ # ping www.baidu.com　　　　　　　　　　　　@容器内可以外网通讯
PING www.baidu.com (110.242.68.4): 56 data bytes
64 bytes from 110.242.68.4: seq=0 ttl=127 time=78.321 ms
64 bytes from 110.242.68.4: seq=1 ttl=127 time=84.424 ms
^C
--- www.baidu.com ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 78.321/81.372/84.424 ms

在overlay网络中，容器想对外提供服务的话，需要在创建容器的过程中配置端口映射

 容器跨节点通讯时的网络原理学习，这里需要注意的是，容器间的隔离是通过namespace实现，但是通过ip netns只能查看到/var/run/netns下的内容，而docker网络中的net namespace是在/var/run/docker/netns，此处需要配置软链接实现ns查看。

[root@docker-1-26 netns]# ln -s /var/run/docker/netns/ /var/run/netns

[root@docker-1-26 ~]# ip netns
e1266b49bcf0 (id: 2)
2-d3f0d488f3 (id: 1)　　　　#node1与node2具有一个相同的net namespace
82a8fe4607bc (id: 0)
e9c0045a3c42
5b2dfc44066a
1bbd1db96893

[root@docker-2-27 ~]# ip netns
34db003b2f09 (id: 2)
1-d3f0d488f3 (id: 1)
8dbb8ea87eb6 (id: 0)
f6ed7fa0e3dd
6422b842d486

[root@docker-1-26 ~]# ip netns exec 2-d3f0d488f3 /bin/bash　　#查看网络命名空间
[root@docker-1-26 ~]# ifconfig

[root@docker-1-26 ~]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP
link/ether 26:01:b8:b0:dd:37 brd ff:ff:ff:ff:ff:ff
inet 20.0.0.254/24 brd 20.0.0.255 scope global br0
valid_lft forever preferred_lft forever
19: vxlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master br0 state UNKNOWN
link/ether 26:01:b8:b0:dd:37 brd ff:ff:ff:ff:ff:ff link-netnsid 0　　#正式因为vxlan网卡属于netnsid0,netns id 0是桥接网络，所以能够在ens间构建vxlan隧道。
21: veth0@if20: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master br0 state UP
link/ether d6:0e:6e:a1:17:dd brd ff:ff:ff:ff:ff:ff link-netnsid 1

技术图片

 

 

 技术图片

 

 容器跨节点通讯的原理图，其实vxlan隧道是构建于node节点 ens网卡间

 

最后是附上两个节点的IP信息

[root@docker-2-27 ~]# docker ps -l
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
415eb4c90bae        busybox:latest      "sh"                22 minutes ago      Up 22 minutes                           busybox_2
[root@docker-2-27 ~]# docker exec -it 415 sh
/ # ip add 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
11: eth0@if12: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue 
    link/ether 02:42:14:00:00:01 brd ff:ff:ff:ff:ff:ff
    inet 20.0.0.1/24 brd 20.0.0.255 scope global eth0
       valid_lft forever preferred_lft forever
13: eth1@if14: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue 
    link/ether 02:42:ac:12:00:02 brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.2/16 brd 172.18.255.255 scope global eth1
       valid_lft forever preferred_lft forever
/ # read escape sequence
[root@docker-2-27 ~]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:a8:31:57 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.27/24 brd 10.0.0.255 scope global ens32
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fea8:3157/64 scope link 
       valid_lft forever preferred_lft forever
3: docker_gwbridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP 
    link/ether 02:42:6b:a0:35:93 brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.1/16 brd 172.18.255.255 scope global docker_gwbridge
       valid_lft forever preferred_lft forever
    inet6 fe80::42:6bff:fea0:3593/64 scope link 
       valid_lft forever preferred_lft forever
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN 
    link/ether 02:42:e8:91:f8:d1 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:e8ff:fe91:f8d1/64 scope link 
       valid_lft forever preferred_lft forever
14: veth8d7f1de@if13: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker_gwbridge state UP 
    link/ether be:9a:e7:28:16:60 brd ff:ff:ff:ff:ff:ff link-netnsid 1
    inet6 fe80::bc9a:e7ff:fe28:1660/64 scope link 
       valid_lft forever preferred_lft forever
[root@docker-2-27 ~]#
技术图片 
10.0.0.26
[root@docker-1-26 ~]# docker ps 
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                                                            NAMES
aa4941dd127a        busybox             "sh"                     15 minutes ago      Up 14 minutes                                                                                        busybox_1
4b8357352d40        progrium/consul     "/bin/start -server …"   23 minutes ago      Up 22 minutes       53/tcp, 53/udp, 8300-8302/tcp, 8400/tcp, 8301-8302/udp, 0.0.0.0:8500->8500/tcp   consule
[root@docker-1-26 ~]# docker exec -it aa4941dd127a sh
/ # ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
15: eth0@if16: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue 
    link/ether 02:42:14:00:00:02 brd ff:ff:ff:ff:ff:ff
    inet 20.0.0.2/24 brd 20.0.0.255 scope global eth0
       valid_lft forever preferred_lft forever
17: eth1@if18: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue 
    link/ether 02:42:ac:12:00:02 brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.2/16 brd 172.18.255.255 scope global eth1
       valid_lft forever preferred_lft forever
/ # read escape sequence
[root@docker-1-26 ~]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:dc:55:5d brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.26/24 brd 10.0.0.255 scope global ens32
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fedc:555d/64 scope link 
       valid_lft forever preferred_lft forever
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP 
    link/ether 02:42:75:ef:c0:4a brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:75ff:feef:c04a/64 scope link 
       valid_lft forever preferred_lft forever
4: docker_gwbridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP 
    link/ether 02:42:40:35:5d:89 brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.1/16 brd 172.18.255.255 scope global docker_gwbridge
       valid_lft forever preferred_lft forever
    inet6 fe80::42:40ff:fe35:5d89/64 scope link 
       valid_lft forever preferred_lft forever
12: vethcfbfeaa@if11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP 
    link/ether 36:29:24:4b:b5:56 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::3429:24ff:fe4b:b556/64 scope link 
       valid_lft forever preferred_lft forever
18: veth5239bf9@if17: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker_gwbridge state UP 
    link/ether 8a:08:e0:f8:ec:82 brd ff:ff:ff:ff:ff:ff link-netnsid 2
    inet6 fe80::8808:e0ff:fef8:ec82/64 scope link 
       valid_lft forever preferred_lft forever
[root@docker-1-26 ~]#
View Code

 

九、docker的跨主机通讯之overlay（vxlan）

标签：messages   hit   display   spl   oop   端口   unix   防火   类型   

原文地址：https://www.cnblogs.com/woshinidaye123/p/14100104.html
踩
(0)
赞
(0)
   
举报
评论 一句话评论（0）
登录后才能评论！
分享档案
更多>
2021年07月29日 (22)
2021年07月28日 (40)
2021年07月27日 (32)
2021年07月26日 (79)
2021年07月23日 (29)
2021年07月22日 (30)
2021年07月21日 (42)
2021年07月20日 (16)
2021年07月19日 (90)
2021年07月16日 (35)
周排行
mamicode.com排行更多图片
更多
友情链接
兰亭集智   国之画   百度统计   站长统计   阿里云   chrome插件   新版天听网
关于我们 - 联系我们 - 留言反馈
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码！