标签：消息队列
openstack的控制节点和计算节点重启后，在控制节点上查看计算资源的状态：
[root@linux-node1 ~]# openstack compute service list +----+------------------+------------------------+----------+---------+-------+---------------------- | ID | Binary | Host | Zone | Status | State | Updated At | +----+------------------+------------------------+----------+---------+-------+---------------------- | 1 | nova-conductor | linux-node1.wanwan.com | internal | enabled | up | 2017-03-10T03:00:40.000000 | | 2 | nova-scheduler | linux-node1.wanwan.com | internal | enabled | up | 2017-03-10T03:00:41.000000 | | 3 | nova-consoleauth | linux-node1.wanwan.com | internal | enabled | up | 2017-03-10T03:00:45.000000 | | 7 | nova-compute | linux-node1.wanwan.com | nova | enabled | up | 2017-03-10T03:00:38.000000 | | 8 | nova-compute | linux-node2.wanwan.com | nova | enabled | down | 2017-03-10T02:28:39.000000 | +----+------------------+------------------------+----------+---------+-------+----------------------
居然发现计算节点node02没有起来，如上在计算节点上查看下状态
[root@linux-node2 ~]# systemctl status openstack-nova-compute.service ● openstack-nova-compute.service - OpenStack Nova Compute Server Loaded: loaded (/usr/lib/systemd/system/openstack-nova-compute.service; enabled; vendor preset: disabled) Active: activating (start) since Fri 2017-03-10 10:49:08 CST; 12min ago Main PID: 2261 (nova-compute) CGroup: /system.slice/openstack-nova-compute.service └─2261 /usr/bin/python2 /usr/bin/nova-compute Mar 10 10:49:08 linux-node2.wanwan.com systemd[1]: Starting OpenStack Nova Compute Server... [root@linux-node2 ~]# systemctl start openstack-nova-compute.service 启服务，发现一直处于卡住的状态，查看下日志文件 -f101b84fa432] AMQP server on 10.10.10.11:5672 is unreachable: [Errno 113] EHOSTUNREACH. Trying again in 32 seconds. Client port: None 2017-03-10 10:58:19.846 2261 ERROR oslo.messaging._drivers.impl_rabbit [req-187b6306-0117-4c40-944d-53dd8bf054be - - - - -] [16162492-f445-4cb0-ad0e-f101b84fa432] AMQP server on 10.10.10.11:5672 is unreachable: [Errno 113] EHOSTUNREACH. Trying again in 32 seconds. Client port: None 2017-03-10 10:58:51.944 2261 ERROR oslo.messaging._drivers.impl_rabbit [req-187b6306-0117-4c40-944d-53dd8bf054be - - - - -] [16162492-f445-4cb0-ad0e-f101b84fa432] AMQP server on 10.10.10.11:5672 is unreachable: [Errno 113] EHOSTUNREACH. Trying again in 32 seconds. Client port: None 2017-03-10 10:59:24.076 2261 ERROR oslo.messaging._drivers.impl_rabbit [req-187b6306-0117-4c40-944d-53dd8bf054be - - - - -] [16162492-f445-4cb0-ad0e-f101b84fa432] AMQP server on 10.10.10.11:5672 is unreachable: [Errno 113] EHOSTUNREACH. Trying again in 32 seconds. Client port: None 2017-03-10 10:59:56.191 2261 ERROR oslo.messaging._drivers.impl_rabbit [req-187b6306-0117-4c40-944d-53dd8bf054be - - - - -] [16162492-f445-4cb0-ad0e-f101b84fa432] AMQP server on 10.10.10.11:5672 is unreachable: [Errno 113] EHOSTUNREACH. Trying again in 32 seconds. Client port: None 2017-03-10 11:00:28.302 2261 ERROR oslo.messaging._drivers.impl_rabbit [req-187b6306-0117-4c40-944d-53dd8bf054be - - - - -] [16162492-f445-4cb0-ad0e-f101b84fa432] AMQP server on 10.10.10.11:5672 is unreachable: [Errno 113] EHOSTUNREACH. Trying again in 32 seconds. Client port: None 2017-03-10 11:01:00.411 2261 ERROR oslo.messaging._drivers.impl_rabbit [req-187b6306-0117-4c40-944d-53dd8bf054be - - - - -] [16162492-f445-4cb0-ad0e-f101b84fa432] AMQP server on 10.10.10.11:5672 is unreachable: [Errno 113] EHOSTUNREACH. Trying again in 32 seconds. Client port: None 2017-03-10 11:01:33.521 2261 ERROR oslo.messaging._drivers.impl_rabbit [req-187b6306-0117-4c40-944d-53dd8bf054be - - - - -] [16162492-f445-4cb0-ad0e-f101b84fa432] AMQP server on 10.10.10.11:5672 is unreachable: [Errno 113] EHOSTUNREACH. Trying again in 32 seconds. Client port: None
提示AMQP不可达，这个时候我就开始怀疑消息队列是否正常，继续检查
[root@linux-node1 ~]# lsof -i :5672 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME nova-cons 1171 nova 5u IPv4 30613 0t0 TCP linux-node1:40614->linux-node1:amqp (ESTABLISHED) beam.smp 1173 rabbitmq 52u IPv6 29124 0t0 TCP *:amqp (LISTEN) beam.smp 1173 rabbitmq 53u IPv6 31152 0t0 TCP linux-node1:amqp->linux-node1:40614 (ESTABLISHED) beam.smp 1173 rabbitmq 54u IPv6 31176 0t0 TCP linux-node1:amqp->linux-node1:40624 (ESTABLISHED) beam.smp 1173 rabbitmq 55u IPv6 31180 0t0 TCP linux-node1:amqp->linux-node1:40626 (ESTABLISHED) beam.smp 1173 rabbitmq 56u IPv6 31183 0t0 TCP linux-node1:amqp->linux-node1:40628 (ESTABLISHED) beam.smp 1173 rabbitmq 57u IPv6 31193 0t0 TCP linux-node1:amqp->linux-node1:40630 (ESTABLISHED) beam.smp 1173 rabbitmq 58u IPv6 31197 0t0 TCP linux-node1:amqp->linux-node1:40632 (ESTABLISHED) beam.smp 1173 rabbitmq 59u IPv6 31255 0t0 TCP linux-node1:amqp->linux-node1:40640 (ESTABLISHED) beam.smp 1173 rabbitmq 60u IPv6 31321 0t0 TCP linux-node1:amqp->linux-node1:40646 (ESTABLISHED) beam.smp 1173 rabbitmq 61u IPv6 31355 0t0 TCP linux-node1:amqp->linux-node1:40654 (ESTABLISHED) beam.smp 1173 rabbitmq 62u IPv6 35079 0t0 TCP linux-node1:amqp->linux-node1:40670 (ESTABLISHED) nova-sche 1186 nova 7u IPv4 31192 0t0 TCP linux-node1:40630->linux-node1:amqp (ESTABLISHED) nova-comp 2091 nova 4u IPv4 31168 0t0 TCP linux-node1:40624->linux-node1:amqp (ESTABLISHED) nova-comp 2091 nova 5u IPv4 31179 0t0 TCP linux-node1:40626->linux-node1:amqp (ESTABLISHED) nova-comp 2091 nova 21u IPv4 31898 0t0 TCP linux-node1:40654->linux-node1:amqp (ESTABLISHED) nova-comp 2091 nova 22u IPv4 35882 0t0 TCP linux-node1:40670->linux-node1:amqp (ESTABLISHED) nova-cond 3265 nova 7u IPv4 31196 0t0 TCP linux-node1:40632->linux-node1:amqp (ESTABLISHED) nova-cond 3265 nova 8u IPv4 31833 0t0 TCP linux-node1:40646->linux-node1:amqp (ESTABLISHED) nova-cond 3267 nova 7u IPv4 30623 0t0 TCP linux-node1:40628->linux-node1:amqp (ESTABLISHED) nova-cond 3267 nova 8u IPv4 31750 0t0 TCP linux-node1:40640->linux-node1:amqp (ESTABLISHED) 没发现有异常，尝试清除下iptables [root@linux-node1 ~]# iptables -F [root@linux-node1 ~]# iptables -X [root@linux-node1 ~]# iptables -Z 再次进行检查 [root@linux-node1 ~]# openstack compute service list +----+------------------+------------------------+----------+---------+-------+---------------------- | ID | Binary | Host | Zone | Status | State | Updated At | +----+------------------+------------------------+----------+---------+-------+---------------------- | 1 | nova-conductor | linux-node1.wanwan.com | internal | enabled | up | 2017-03-10T03:08:40.000000 | | 2 | nova-scheduler | linux-node1.wanwan.com | internal | enabled | up | 2017-03-10T03:08:41.000000 | | 3 | nova-consoleauth | linux-node1.wanwan.com | internal | enabled | up | 2017-03-10T03:08:45.000000 | | 7 | nova-compute | linux-node1.wanwan.com | nova | enabled | up | 2017-03-10T03:08:48.000000 | | 8 | nova-compute | linux-node2.wanwan.com | nova | enabled | up | 2017-03-10T03:08:40.000000 | +----+------------------+------------------------+----------+---------+-------+----------------------
如上，可以发现计算节点已经恢复正常了，看来iptables一定要记得清空策略
本文出自 “冰冻vs西瓜” 博客，请务必保留此出处http://molewan.blog.51cto.com/287340/1905064
标签：消息队列
原文地址：http://molewan.blog.51cto.com/287340/1905064
