标签：cut   inxi   protoc   word   type=file   desc   roo   完成   dap   

1概述

顶新苏州IDC机房配置了四个Linux HA+ Oracle数据库集群。其配置信息如下：

1.1店会系统&FC-arap系统集群

1.2订购-营收&主档-预算检讨-基盘集群

1.3IDC-SCM系统集群

集群另外一个节点是10.0.101.3。

1.4JDE系统&JDE测试系统集群

2安装配置手册

RHEL 6.8 HA和RHEL 7.2 HA的安装过程不相同，但是集群的配置过程相同。我们将软件安装过程分开，配置过程放在一起描述。

2.1       . RHEL 6.8 HA软件安装

将RHEL 6.8 ISO文件挂载到/software/rh68目录。编辑/etc/yum.conf文件。

2.1.1  编辑/etc/yum.conf

[local]

name=local

baseurl=file:///software/rh68

enabled=1

gpgcheck=1

gpgkey=file:///software/rh68/RPM-GPG-KEY-redhat-release

[HighAvailability]

name=HighAvailability

baseurl=file:///software/rh68/HighAvailability

enabled=1

gpgcheck=0

2.1.2  安装软件

# yum install pcs pacemaker cman fence-agents

从RHEL 6.5以后，RHEL的HA通过pcs来管理，不再通过rgmanager管理。只有RHEL 6.8才有Oracle的Agent。

2.1.3  禁用和停止主机防火墙服务

禁用防火墙

#chkconfig iptables off

#chkconfig ip6tables off

停止防火墙服务

#service iptables stop

#service ip6tables off

For TCP: Ports 2224, 3121, 21064

For UDP: Ports, 5405

2.1.4  启动和配置服务pcsd

# chkconfig  pcsd on

# service pcsd start

2.1.5  重置hacluster密码

hacluster是pcs 管理员账号，建议集群中的服务器设置相同的密码。

# passwd hacluster (hacluster1234)

更改用户 hacluster 的密码 。

新的 密码：

无效的密码： 密码包含用户名在某些地方

重新输入新的 密码：

passwd：所有的身份验证令牌已经成功更新。

2.2RHEL 7.2 HA软件安装

2.2.1  /etc/yum.conf

[HighAvailability]

name=HighAvailability

baseurl=file:///software/rh72/addons/HighAvailability

enabled=1

gpgcheck=0

[local]

name=local

baseurl=file:///software/rh72/

enabled=1

gpgcheck=1

gpgkey=file:///software/rh72/RPM-GPG-KEY-redhat-release

2.2.2  安装软件

# yum install pcs fence-agents-all

只有RHEL 7.2才有Oracle的HA Agent, RHEL 7.0和7.1都没有Oracle的HA Agent。在RHEL 7.2,将不再使用cman服务.

2.2.3  启动和配置服务pcsd

[root@fmprfcdb ~]# systemctl start pcsd.service

[root@fmprfcdb ~]# systemctl enable pcsd.service

ln -s ‘/usr/lib/systemd/system/pcsd.service‘ ‘/etc/systemd/system/multi-user.target.wants/pcsd.service‘

检查pcsd服务能随机自动重新启动

[root@fmpracora dinxin]# systemctl list-unit-files |grep pcs

pcsd.service                                enabled

rpcsvcgssd.service                          static

2.2.4  禁用和停止主机防火墙服务

#systemctl stop firewalld

2.2.5  重置hacluster密码

hacluster是pcs 管理员账号，建议集群中的服务器设置相同的密码。

# passwd hacluster (hacluster1234)

更改用户 hacluster 的密码 。

新的 密码：

无效的密码： 密码包含用户名在某些地方

重新输入新的 密码：

passwd：所有的身份验证令牌已经成功更新。

2.3配置集群

RHEL 6.8和RHEL 7.2配置集群的步骤相同。将在以下介绍。

2.3.1  编辑/etc/hosts

[root@fmprpoora oracle]#  cat /etc/hosts

127.0.0.1localhost localhost.localdomain localhost4 localhost4.localdomain4

::1localhost localhost.localdomain localhost6 localhost6.localdomain6

10.0.101.6 fmprmaora

10.0.101.5 fmprpoora

10.0.200.16 fmprpoora-priv

10.0.200.20 fmprmaora-priv

10.0.101.7fmprpoora-vip

10.0.101.8fmprmaora-vip

其中fmprpoora-priv，fmprmaora-priv为心跳IP地址对应的主机名，将用作集群间的通讯。

2.3.2  授权hacluster 访问集群中的节点

# pcs cluster auth fmprfcdb-priv fmpracdb-priv

Username: hacluster

Password:

fmprfcdb-priv: Authorized

fmpracdb-priv: Authorized

授权的令牌保存在以下目录中。

[root@fmprpoora oracle]# cat /var/lib/pcsd/tokens

{

  "format_version": 2,

  "data_version": 3,

  "tokens": {

    "fmprmaora-priv": "a2a19af3-584e-4f88-873b-c3bee27d137c",

    "fmprpoora-priv": "33169580-5ba5-4cb3-9a1c-a4a87fd3d226"

  }

2.3.3  创建cluster

 [root@fmprfcora pcsd]# pcs cluster setup --start --name fmprfcac_cluster  fmprfcora-priv fmpracora-priv

Shutting down pacemaker/corosync services...

Redirecting to /bin/systemctl stop  pacemaker.service

Redirecting to /bin/systemctl stop  corosync.service

Killing any remaining services...

Removing all cluster configuration files...

fmprfcora-priv: Succeeded

fmpracora-priv: Succeeded

Starting cluster on nodes: fmprfcora-priv, fmpracora-priv...

fmpracora-priv: Starting Cluster...

fmprfcora-priv: Starting Cluster...

Synchronizing pcsd certificates on nodes fmprfcora-priv, fmpracora-priv...

fmprfcora-priv: Success

fmpracora-priv: Success

Restaring pcsd on the nodes in order to reload the certificates...

fmprfcora-priv: Success

fmpracora-priv: Success

[root@fmprfcora pcsd]#

2.3.4  配置集群服务随主机启动而自动启动

[root@fmprfcora pcsd]#  pcs cluster enable --all

fmprfcora-priv: Cluster Enabled

fmpracora-priv: Cluster Enabled

2.3.5  检查cluster服务状态

[root@fmpracdb pcsd]# pcs cluster status          

Cluster Status:

 Last updated: Sun Aug  7 11:07:59 2016         Last change: Sun Aug  7 11:06:44 2016 by hacluster via crmd on fmpracora-priv

 Stack: corosync

 Current DC: fmpracora-priv (version 1.1.13-10.el7-44eb2dd) - partition with quorum

 2 nodes and 0 resources configured

 Online: [ fmpracora-priv fmprfcora-priv ]

PCSD Status:

fmprfcora-priv: Online

fmpracora-priv: Online

2.3.6  配置集群的fence设备

#pcs stonith create imm-fmprjdeora fence_ipmilan action=reboot auth=password ipaddr=10.0.202.18 lanplus=1 login=USERID passwd=PASSW0RD delay=120cipher=1 pcmk_host_list=fmprjdeora-priv pcmk_host_check=static-list power_timeout=60

#pcs stonith create imm-fmdrjdeora fence_ipmilan action=reboot auth=password ipaddr=10.0.202.22 lanplus=1  login=USERID passwd=PASSW0RD  cipher=1 pcmk_host_list=fmdrjdeora-priv pcmk_host_check=static-list power_timeout=60

查看当前fence设备配置的命令如下：

[root@fmprpoora oracle]# pcs stonith show --full

 Resource: imm-fmprmaora (class=stonith type=fence_ipmilan)

Attributes: action=reboot auth=password ipaddr=10.0.202.20 lanplus=1 login=USERID passwd=PASSW0RD cipher=1 pcmk_host_list=fmprmaora-priv pcmk_host_check=static-list power_timeout=60

Operations: monitor interval=60s (imm-fmprmaora-monitor-interval-60s)

 Resource: imm-fmprpoora (class=stonith type=fence_ipmilan)

Attributes: action=reboot auth=password ipaddr=10.0.202.16 lanplus=1 login=USERID passwd=PASSW0RD delay=5 cipher=1 pcmk_host_list=fmprpoora-priv pcmk_host_check=static-list power_timeout=60 debug=/tmp/fmprpoora.log

Operations: monitor interval=60s (imm-fmprpoora-monitor-interval-60s)

[root@fmprpoora oracle]#

2.3.7  测试fence

检查能否使用fence_ipmilan命令查看fence设备的状态

[root@fmpridcora oracle]# fence_ipmilan -a 10.0.202.22 -P -l USERID -p PASSW0RD -o status -v

Executing: /usr/bin/ipmitool -I lanplus -H 10.0.202.22 -U USERID -P PASSW0RD -p 623 -L ADMINISTRATOR chassis power status

0 Chassis Power is on

Status: ON

测试能否使用命令行将节点fence，fence之后的设备将自动重新启动

root@fmprpoora ~]# pcs stonith fence fmprmaora-priv

Node: fmprmaora-priv fenced

 [root@fmprmaora ~]#  pcs stonith fence fmprpoora-priv

Node: fmprpoora-priv fenced

2.3.8  创建浮动IP

创建浮动IP地址资源

[root@fmprfcora pcsd]# pcs resource create fmprfcora-vip ocf:heartbeat:IPaddr2 ip=10.0.101.12 cidr_netmask=24 op monitor interval=30s

[root@fmprfcora pcsd]# pcs resource create fmpracora-vip ocf:heartbeat:IPaddr2 ip=10.0.101.11 cidr_netmask=24 op monitor interval=30s  

[root@fmprfcora pcsd]#   pcs resource show  --full

 Resource: fmprfcora-vip (class=ocf provider=heartbeat type=IPaddr2)

Attributes: ip=10.0.101.12 cidr_netmask=24

Operations: start interval=0s timeout=20s (fmprfcora-vip-start-interval-0s)

              stop interval=0s timeout=20s (fmprfcora-vip-stop-interval-0s)

              monitor interval=30s (fmprfcora-vip-monitor-interval-30s)

 Resource: fmpracora-vip (class=ocf provider=heartbeat type=IPaddr2)

Attributes: ip=10.0.101.11 cidr_netmask=24

Operations: start interval=0s timeout=20s (fmpracora-vip-start-interval-0s)

              stop interval=0s timeout=20s (fmpracora-vip-stop-interval-0s)

              monitor interval=30s (fmpracora-vip-monitor-interval-30s)

2.3.9  创建文件系统资源

文件系统和挂载点需要已经创建好，并在集群中的各节点依次挂载完成。接下来在集群的文件系统资源。

[root@fmpracdb pcsd]# pcs resource create ac_fs Filesystem \

>device="/dev/mapper/acvg-aclv" directory="/ac-oradata" fstype="ext4"

[root@fmpracdb pcsd]#    pcs resource create fc_fs Filesystem \

>device="/dev/mapper/fcvg-fclv" directory="/fc-oradata" fstype="ext4"

显示文件系统资源的配置

[root@fmpracora dinxin]# pcs resource show ac_fs

 Resource: ac_fs (class=ocf provider=heartbeat type=Filesystem)

Attributes: device=/dev/mapper/acvg-aclv directory=/ac-oradata fstype=ext4

Operations: start interval=0s timeout=60 (ac_fs-start-interval-0s)

              stop interval=0s timeout=60 (ac_fs-stop-interval-0s)

              monitor interval=20 timeout=40 (ac_fs-monitor-interval-20)

[root@fmpracora dinxin]# pcs resource show fc_fs

 Resource: fc_fs (class=ocf provider=heartbeat type=Filesystem)

Attributes: device=/dev/mapper/fcvg-fclv directory=/fc-oradata fstype=ext4

Operations: start interval=0s timeout=60 (fc_fs-start-interval-0s)

              stop interval=0s timeout=60 (fc_fs-stop-interval-0s)

              monitor interval=20 timeout=40 (fc_fs-monitor-interval-20)

2.3.10 创建数据库lsnr和db资源

确认当前pcs版本支持Oracle数据库。以下表示支持oracle数据库

[root@fmpracora dinxin]# pcs resource list  |grep ora

ocf:heartbeat:oracle - Manages an Oracle Database instance

ocf:heartbeat:oralsnr - Manages an Oracle TNS listener

确认oracle数据库和listener已经创建好，并且手工能够在集群中的节点正常启动到open状态。

确保能解析数据库sid,

[oracle@fmpracora admin]$ tnsping cvs

TNS Ping Utility for Linux: Version 11.2.0.4.0 - Production on 23-AUG-2016 13:51:06

Copyright (c) 1997, 2013, Oracle.  All rights reserved.

Used parameter files:

Used TNSNAMES adapter to resolve the alias

Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = fmpracora-vip)(PORT = 1526)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = cvs)))

OK (0 msec)

[oracle@fmpracora admin]$ tnsping arap2

TNS Ping Utility for Linux: Version 11.2.0.4.0 - Production on 23-AUG-2016 13:51:10

Copyright (c) 1997, 2013, Oracle.  All rights reserved.

Used parameter files:

Used TNSNAMES adapter to resolve the alias

Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = fmprfcora-vip)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = arap2)))

OK (0 msec)

当前的oracle listener.ora和tnsnames.ora文件如下：

CVS =

(DESCRIPTION =

(ADDRESS = (PROTOCOL = TCP)(HOST = fmpracora-vip)(PORT = 1526))

(CONNECT_DATA =

(SERVER = DEDICATED)

(SERVICE_NAME = cvs)

    )

  )

ARAP2 =

(DESCRIPTION =

(ADDRESS = (PROTOCOL = TCP)(HOST = fmprfcora-vip)(PORT = 1521))

(CONNECT_DATA =

(SERVER = DEDICATED)

(SERVICE_NAME = arap2)

    )

  )

[oracle@fmpracora admin]$ cat listener.ora

LSNR_arap2=

(ADDRESS_LIST =

(ADDRESS = (PROTOCOL = TCP)(HOST = fmprfcora-vip)(PORT = 1521))

  )

SECURE_REGISTER_LSNR_arap2 = (TCP)

LSNR_cvs=

(ADDRESS_LIST =

(ADDRESS = (PROTOCOL = TCP)(HOST = fmpracora-vip)(PORT = 1526))

  )

SECURE_REGISTER_LSNR_cvs = (TCP)

使用以下命令创建oracle数据库资源和listnener资源

[root@fmpracdb pcsd]# pcs resource create  arap2db oracle sid=arap2 home=/home/app/11.2/db_1 user=oracle shutdown_method=immediate    

[root@fmpracdb pcsd]# pcs resource create  arap2_lsn oralsnr sid=arap2  home=/home/app/11.2/db_1 user=oracle listener=LSNR_arap2  

[root@fmpracdb pcsd]# pcs resource create  cvsdb oracle sid=cvs home=/home/app/11.2/db_1 user=oracle shutdown_method=immediate    

[root@fmpracdb pcsd]# pcs resource create  cvs_lsn oralsnr sid=cvs  home=/home/app/11.2/db_1 user=oracle listener=LSNR_cvs2

使用以下命令查看当前数据库资源和listener资源配置

[root@fmpracora dinxin]# pcs resource show cvs_lsn

 Resource: cvs_lsn (class=ocf provider=heartbeat type=oralsnr)

Attributes: sid=cvs home=/home/app/11.2/db_1 user=oracle listener=LSNR_cvs

Operations: start interval=0s timeout=120 (cvs_lsn-start-interval-0s)

              stop interval=0s timeout=120 (cvs_lsn-stop-interval-0s)

              monitor interval=10 timeout=30 (cvs_lsn-monitor-interval-10)

[root@fmpracora dinxin]# pcs resource show cvsdb

 Resource: cvsdb (class=ocf provider=heartbeat type=oracle)

Attributes: sid=cvs home=/home/app/11.2/db_1 user=oracle shutdown_method=immediate

Operations: start interval=0s timeout=120 (cvsdb-start-interval-0s)

              stop interval=0s timeout=120 (cvsdb-stop-interval-0s)

              monitor interval=120 timeout=30 (cvsdb-monitor-interval-120)

[root@fmpracora dinxin]# pcs resource show arap2db

 Resource: arap2db (class=ocf provider=heartbeat type=oracle)

Attributes: sid=arap2 home=/home/app/11.2/db_1 user=oracle shutdown_method=immediate

Operations: start interval=0s timeout=120 (arap2db-start-interval-0s)

              stop interval=0s timeout=120 (arap2db-stop-interval-0s)

              monitor interval=120 timeout=30 (arap2db-monitor-interval-120)

[root@fmpracora dinxin]# pcs resource show arap2_lsn

 Resource: arap2_lsn (class=ocf provider=heartbeat type=oralsnr)

Attributes: sid=arap2 home=/home/app/11.2/db_1 user=oracle listener=LSNR_arap2

Operations: start interval=0s timeout=120 (arap2_lsn-start-interval-0s)

              stop interval=0s timeout=120 (arap2_lsn-stop-interval-0s)

              monitor interval=10 timeout=30 (arap2_lsn-monitor-interval-10)

[root@fmpracora dinxin]#

2.3.11 创建集群组

创建集群组，在组中添加浮动IP地址资源，文件系统资源，oracle listener资源，以及Oracle数据库资源，并设置资源间的依赖关系。组设置成功后，将作为一个整体进行节点间的切换。

# pcs resource group add  ac_group fmpracora-vip

#pcs resource group add  ac_group ac_fs --after fmpracora-vip

#pcs resource group add  ac_group cvs_lsn --after ac_fs

#pcs resource group add  ac_group cvsdb --after cvs_lsn

2.3.12 创建集群资源优先启动节点

集群节点将在优先节点启动。当优先节点因故障关机后，自动切换到其它节点。在优先节点开机重新启动后，将跟随cman,pcs服务启动而自动回切到优先节点。

#pcs constraint location fc_fs   prefers fmprfcora-priv

#pcs constraint location fmprfcora-vip prefers fmprfcora-priv

#pcs constraint location  cvs_lsn prefers fmprfcora-priv

#pcs constraint location  cvsdb prefers fmprfcora-priv

#pcs constraint location  fc_group prefers fmprfcora-priv

#pcs constraint location ac_fs   prefers fmpracora-priv

#pcs constraint location fmpracora-vip prefers fmpracora-priv

#pcs constraint location  apar2_lsn  prefers fmpracora-priv

#pcs constraint location  apar2db prefers fmpracora-priv

#pcs constraint location  ac_group prefers fmpracora-priv

查看优先节点：

[root@fmpracora dinxin]# pcs constraint

Location Constraints:

  Resource: ac_group

    Enabled on: fmpracora-priv (score:INFINITY)

  Resource: fc_group

    Enabled on: fmprfcora-priv (score:INFINITY)

  Resource: fmpracora-vip

    Enabled on: fmpracora-priv (score:INFINITY)

  Resource: fmprfcora-vip

    Enabled on: fmprfcora-priv (score:INFINITY)

3RHEL 6.8 & 7.2 HA维护手册

3.1移动资源组到其它节点

[root@fmdrjdeora ~]#  pcs resource move dr_group

Warning: Creating location constraint cli-ban-dr_group-on-fmdrjdeora-priv with a score of -INFINITY for resource dr_group on node fmdrjdeora-priv.

This will prevent dr_group from running on fmdrjdeora-priv until the constraint is removed. This will be the case even if fmdrjdeora-priv is the last node in the cluster.

服务迁移到其它节点。

[root@fmdrjdeora ~]# pcs resource

 Resource Group: dr_group

fmdrjdeora-vip(ocf::heartbeat:IPaddr2):Started fmprjdeora-priv

dr_fs(ocf::heartbeat:Filesystem):Started fmprjdeora-priv

jdedbdr_lsn(ocf::heartbeat:oralsnr):Started fmprjdeora-priv

jdedr_db(ocf::heartbeat:oracle):Started fmprjdeora-priv

 Resource Group: pr_group

pr_fs(ocf::heartbeat:Filesystem):Started fmprjdeora-priv

fmprjdeora-vip(ocf::heartbeat:IPaddr2):Started fmprjdeora-priv

jdedb(ocf::heartbeat:oracle):Started fmprjdeora-priv

jde_lsn(ocf::heartbeat:oralsnr):Started fmprjdeora-priv

且生成一个在节点fmdrjdeora-priv禁用

[root@fmdrjdeora oracle]# pcs constraint

Location Constraints:

Resource: dr_fs

Enabled on: fmdrjdeora-priv (score:INFINITY)

Resource: dr_group

Enabled on: fmdrjdeora-priv (score:300)

Enabled on: fmprjdeora-priv (score:100)

Disabled on: fmdrjdeora-priv (score:-INFINITY) (role: Started)

3.2回切该资源回到原优先节点

需要清除限制，否则该资源不会fmdrjdeora-priv运行。

[root@fmprjdeora ~]#  pcs resource clear dr_group

清除资源后，constraint恢复正常。

[root@fmprjdeora ~]# pcs constraint

Location Constraints:

Resource: dr_fs

Enabled on: fmdrjdeora-priv (score:INFINITY)

Resource: dr_group

Enabled on: fmdrjdeora-priv (score:300)

Enabled on: fmprjdeora-priv (score:100)

Resource: fmdrjdeora-vip

Enabled on: fmdrjdeora-priv (score:300)

Resource: fmprjdeora-vip

Enabled on: fmprjdeora-priv (score:INFINITY)

Resource: jde_lsn

Enabled on: fmprjdeora-priv (score:INFINITY)

Resource: jdedb

Enabled on: fmprjdeora-priv (score:INFINITY)

Resource: pr_fs

Enabled on: fmprjdeora-priv (score:INFINITY)

Resource: pr_group

Enabled on: fmprjdeora-priv (score:INFINITY)

Ordering Constraints:

Resource Sets:

set fmprjdeora-vip pr_fs jde_lsn jdedb

set fmdrjdeora-vip dr_fs

Colocation Constraints:

资源就会自动回切到原始服务器。

[root@fmdrjdeora ~]# pcs resource clear dr_group

[root@fmdrjdeora ~]# pcs resource

 Resource Group: dr_group

fmdrjdeora-vip(ocf::heartbeat:IPaddr2):Started fmdrjdeora-priv

dr_fs(ocf::heartbeat:Filesystem):Stopped

jdedbdr_lsn(ocf::heartbeat:oralsnr):Stopped

jdedr_db(ocf::heartbeat:oracle):        Stopped

 Resource Group: pr_group

pr_fs(ocf::heartbeat:Filesystem):Started fmprjdeora-priv

fmprjdeora-vip(ocf::heartbeat:IPaddr2):Started fmprjdeora-priv

jdedb(ocf::heartbeat:oracle):Started fmprjdeora-priv

jde_lsn(ocf::heartbeat:oralsnr):Started fmprjdeora-priv

[root@fmdrjdeora ~]# pcs resource

 Resource Group: dr_group

fmdrjdeora-vip(ocf::heartbeat:IPaddr2):Started fmdrjdeora-priv

dr_fs(ocf::heartbeat:Filesystem):Started fmdrjdeora-priv

jdedbdr_lsn(ocf::heartbeat:oralsnr):Started fmdrjdeora-priv

jdedr_db(ocf::heartbeat:oracle):Started fmdrjdeora-priv

 Resource Group: pr_group

pr_fs(ocf::heartbeat:Filesystem):    Started fmprjdeora-priv

fmprjdeora-vip(ocf::heartbeat:IPaddr2):Started fmprjdeora-priv

jdedb(ocf::heartbeat:oracle):Started fmprjdeora-priv

jde_lsn(ocf::heartbeat:oralsnr):Started fmprjdeora-priv

3.3查看资源运行状态

[root@fmpracora dinxin]# pcs resource

 Resource Group: ac_group

fmpracora-vip(ocf::heartbeat:IPaddr2):Started fmpracora-priv

ac_fs(ocf::heartbeat:Filesystem):Started fmpracora-priv

cvs_lsn(ocf::heartbeat:oralsnr):       Started fmpracora-priv

cvsdb(ocf::heartbeat:oracle):Started fmpracora-priv

 Resource Group: fc_group

fmprfcora-vip(ocf::heartbeat:IPaddr2):Started fmprfcora-priv

fc_fs(ocf::heartbeat:Filesystem):Started fmprfcora-priv

arap2_lsn(ocf::heartbeat:oralsnr):Started fmprfcora-priv

arap2db(ocf::heartbeat:oracle):Started fmprfcora-priv

[root@fmpracora dinxin]#

3.4查看资源运行状态

[root@fmpracora dinxin]# pcs status

Cluster name: fmprfcac_cluster

Last updated: Tue Aug 23 15:29:07 2016          Last change: Tue Aug 23 15:24:26 2016 by root via cibadmin on fmpracora-priv

Stack: corosync

Current DC: fmpracora-priv (version 1.1.13-10.el7-44eb2dd) - partition with quorum

2 nodes and 10 resources configured

Online: [ fmpracora-priv fmprfcora-priv ]

Full list of resources:

 imm-fmprfcora(stonith:fence_ipmilan):Started fmpracora-priv

 imm-fmpracora(stonith:fence_ipmilan):Started fmprfcora-priv

 Resource Group: ac_group

fmpracora-vip(ocf::heartbeat:IPaddr2):Started fmpracora-priv

ac_fs(ocf::heartbeat:Filesystem):Started fmpracora-priv

cvs_lsn(ocf::heartbeat:oralsnr):Started fmpracora-priv

cvsdb(ocf::heartbeat:oracle):        Started fmpracora-priv

 Resource Group: fc_group

fmprfcora-vip(ocf::heartbeat:IPaddr2):Started fmprfcora-priv

fc_fs(ocf::heartbeat:Filesystem):Started fmprfcora-priv

arap2_lsn(ocf::heartbeat:oralsnr):Started fmprfcora-priv

arap2db(ocf::heartbeat:oracle):Started fmprfcora-priv

Failed Actions:

* cvsdb_monitor_120000 on fmpracora-priv ‘not running‘ (7): call=245, status=complete, exitreason=‘none‘,

last-rc-change=‘Mon Aug 22 14:05:39 2016‘, queued=0ms, exec=0ms

* cvs_lsn_monitor_10000 on fmpracora-priv ‘not running‘ (7): call=243, status=complete, exitreason=‘none‘,

last-rc-change=‘Tue Aug 23 10:45:13 2016‘, queued=0ms, exec=0ms

PCSD Status:

fmprfcora-priv: Online

fmpracora-priv: Online

Daemon Status:

corosync: active/enabled

pacemaker: active/enabled

pcsd: active/enabled

3.5standby节点

设置节点为standby，该节点的所有集群资源将自动切换到集群中的其它节点

[root@fmprpoora ~]# pcs cluster standby fmprpoora-priv

取消standby的命令是unstandby，该节点的所有集群资源将自动回切到本节点。

[root@fmprpoora ~]# pcs cluster unstandby fmprpoora-priv

3.6清除资源错误计数计

[root@fmpracora dinxin]# pcs resource cleanup

Waiting for 1 replies from the CRMd. OK

3.7维护节点

节点处于维护状态时，该节点上的资源将不会自动启动。

3.7.1  RHEL 6.8维护节点

[root@fmpridcora oracle]#pcs node maintenance fmpridcora-priv

[root@fmpridcora oracle]# pcs status

Cluster name: fmpridc-clu

Last updated: Tue Aug 23 14:56:21 2016          Last change: Tue Aug 23 14:55:59                2016 by root via crm_attribute on fmpridcora-priv

Stack: cman

Current DC: fmprbiora-priv (version 1.1.14-8.el6-70404b0) - partition with quoru               m

2 nodes and 6 resources configured

Node fmpridcora-priv: maintenance

Online: [ fmprbiora-priv ]

Full list of resources:

 imm-fmpridcora (stonith:fence_ipmilan):        Started fmprbiora-priv

 imm-fmprbiora(stonith:fence_ipmilan):Started fmpridcora-priv (unmanag ed)

 Resource Group: idc_group

     fmpridcora-vip     (ocf::heartbeat:IPaddr2):   Started fmpridcora-priv      (unmanaged)

     idc_fs(ocf::heartbeat:Filesystem):    Started fmpridcora-priv (unmanaged)

     fmidc_lsn(ocf::heartbeat:oralsnr):Started fmpridcora-priv (unmanaged)

     fmidcdb(ocf::heartbeat:oracle):        Started fmpridcora-priv (unmanaged)

PCSD Status:

fmpridcora-priv: Online

fmprbiora-priv: Online

清除节点维护状态：

[root@fmpridcora oracle]# pcs node unmaintenance fmpridcora-priv

 [root@fmpridcora oracle]# pcs status

Cluster name: fmpridc-clu

Last updated: Tue Aug 23 15:32:28 2016          Last change: Tue Aug 23 15:31:19 2016 by root via crm_attribute on fmpridcora-priv

Stack: cman

Current DC: fmprbiora-priv (version 1.1.14-8.el6-70404b0) - partition with quorum

2 nodes and 6 resources configured

Online: [ fmprbiora-priv fmpridcora-priv ]

Full list of resources:

 imm-fmpridcora (stonith:fence_ipmilan):        Started fmprbiora-priv

 imm-fmprbiora(stonith:fence_ipmilan):Started fmpridcora-priv

 Resource Group: idc_group

fmpridcora-vip(ocf::heartbeat:IPaddr2):Started fmpridcora-priv

idc_fs(ocf::heartbeat:Filesystem):Started fmpridcora-priv

fmidc_lsn(ocf::heartbeat:oralsnr):Started fmpridcora-priv

fmidcdb(ocf::heartbeat:oracle):Started fmpridcora-priv

PCSD Status:

fmpridcora-priv: Online

fmprbiora-priv: Online

3.7.2  RHEL 7.2

[root@fmpracora dinxin]# pcs property set maintenance-mode=true

[root@fmpracora dinxin]# pcs property show  maintenance-mode

Cluster Properties:

 maintenance-mode: true

[root@fmpracora dinxin]# pcs status

Cluster name: fmprfcac_cluster

Last updated: Tue Aug 23 15:11:46 2016          Last change: Tue Aug 23 15:09:15 2016 by root via cibadmin on fmpracora-priv

Stack: corosync

Current DC: fmpracora-priv (version 1.1.13-10.el7-44eb2dd) - partition with quorum

2 nodes and 10 resources configured

Online: [ fmpracora-priv fmprfcora-priv ]

Full list of resources:

 imm-fmprfcora(stonith:fence_ipmilan):Started fmpracora-priv (unmanaged)

 imm-fmpracora(stonith:fence_ipmilan):Started fmprfcora-priv (unmanaged)

 Resource Group: ac_group

fmpracora-vip(ocf::heartbeat:IPaddr2):Started fmpracora-priv (unmanaged)

ac_fs(ocf::heartbeat:Filesystem):Started fmpracora-priv (unmanaged)

cvs_lsn(ocf::heartbeat:oralsnr):Stopped (unmanaged)

cvsdb      (ocf::heartbeat:oracle):        Started fmpracora-priv (unmanaged)

 Resource Group: fc_group

fmprfcora-vip(ocf::heartbeat:IPaddr2):Started fmprfcora-priv (unmanaged)

fc_fs(ocf::heartbeat:Filesystem):Started fmprfcora-priv (unmanaged)

     arap2_lsn(ocf::heartbeat:oralsnr):Started fmprfcora-priv (unmanaged)

arap2db(ocf::heartbeat:oracle):Started fmprfcora-priv (unmanaged)

PCSD Status:

fmprfcora-priv: Online

fmpracora-priv: Online

Daemon Status:

corosync: active/enabled

pacemaker: active/enabled

pcsd: active/enabled

清除维护模式

[root@fmpracora dinxin]# pcs property set maintenance-mode=false

清除维护模式后，系统状态恢复正常。

[root@fmpracora dinxin]# pcs resource

 Resource Group: ac_group

fmpracora-vip      (ocf::heartbeat:IPaddr2):       Started fmpracora-priv

ac_fs(ocf::heartbeat:Filesystem):Started fmpracora-priv

cvs_lsn(ocf::heartbeat:oralsnr):Started fmpracora-priv

cvsdb(ocf::heartbeat:oracle):Started fmpracora-priv

 Resource Group: fc_group

fmprfcora-vip(ocf::heartbeat:IPaddr2):Started fmprfcora-priv

fc_fs(ocf::heartbeat:Filesystem):Started fmprfcora-priv

arap2_lsn(ocf::heartbeat:oralsnr):Started fmprfcora-priv

arap2db(ocf::heartbeat:oracle):Started fmprfcora-priv

3.8手工启动资源

当系统处于维护模式时，可以手工启动资源。启动资源加上--full参数，可以看到详细的输出。

[root@fmprpoora ~]# pcs resource debug-start fmprpoora-vip

Operation start for fmprpoora-vip (ocf:heartbeat:IPaddr2) returned 0

 >stderr: INFO: Adding inet address 10.0.101.7/24 with broadcast address 10.0.101.255 to device bond0

 >stderr: INFO: Bringing device bond0 up

 >stderr: INFO: /usr/libexec/heartbeat/send_arp -i 200 -r 5 -p /var/run/resource-agents/send_arp-10.0.101.7 bond0 10.0.101.7 auto not_used not_used

3.9手工停止资源

当系统处于维护模式时，可以停止资源。

[root@fmprmaora ~]# pcs resource debug-stop po_fs

Operation stop for po_fs (ocf:heartbeat:Filesystem) returned 0

 >stderr: INFO: Running stop for /dev/mapper/povg-polv on /po-oradata

 >stderr: INFO: Trying to unmount /po-oradata

 >stderr: INFO: unmounted /po-oradata successfully

[root@fmprmaora ~]# pcs resource debug-stop fmprpoora-vip

Operation stop for fmprpoora-vip (ocf:heartbeat:IPaddr2) returned 0

 >stderr: INFO: IP status = ok, IP_CIP=

[root@fmprmaora ~]#

3.10删除cluster配置

[root@fmprpoora ~]# pcs cluster destroy

Shutting down pacemaker/corosync services...

Pacemaker Cluster Manager is already stopped               [确定]

Stopping cluster:

Leaving fence domain...                                 [确定]

Stopping gfs_controld...                                [确定]

Stopping dlm_controld...                                [确定]

Stopping fenced...                                      [确定]

Stopping cman...                                        [确定]

Unloading kernel modules...                             [确定]

Unmounting configfs...                                  [确定]

Killing any remaining services...

Removing all cluster configuration files...

[root@fmprpoora ~]#

Linux HA安装维护手册

标签：cut   inxi   protoc   word   type=file   desc   roo   完成   dap   

原文地址：https://www.cnblogs.com/liang545621/p/12610484.html