linux下安装drbd详解

时间：2014-09-26 12:56:39 阅读：325 评论：0 收藏：0 [点我收藏+]

标签：des style blog http color io os 使用 ar

关于drbd理论方面的知识本文中不做介绍，本文章主要讲述如何搭建drbd。

一、系统环境

系统平台: Centos6.3(内核2.6.32-431.29.2.el6.x86_64)

节点一主机名: Cenos-HA1 IP:192.168.1.202

节点一主机名: Cenos-HA2 IP:192.168.1.202

DRBD版本：DRBD-8.4.3

二、安装

1.准备

(1).关闭iptables和SELINUX，避免安装过程中报错。

节点1和节点2

#/etc/init.d/iptables stop

#/etc/init.d/ip6tables stop

#chkconfig iptables off

#chkconfig ip6tables off

# vi /etc/sysconfig/selinux

---------------

SELINUX=disabled

---------------

(2).设置 hosts文件(节点1和节点2)

# vi /etc/hosts

-----------------

192.168.1.202 Centos-HA1
192.168.1.203 Centos-HA2

-----------------

节点1和节点2(重启）

(3).在两台虚拟机分别添加一块2G硬盘sdb作为DRBD，分别分区为sdb1，大小2G，并在本地系统创建/data目录，不做挂载操作。(节点1和节点2)

节点1

[root@Centos-HA1 ~]# fdisk /dev/sdb

Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xd59786d2.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won‘t be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It‘s strongly recommended to
         switch off the mode (command ‘c‘) and change display units to
         sectors (command ‘u‘).

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-261, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-261, default 261):
Using default value 261

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

[root@Centos-HA1 ~]#mkdir /data

节点2

[root@Centos-HA2 ~]# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x9adc5d39.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won‘t be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It‘s strongly recommended to
         switch off the mode (command ‘c‘) and change display units to
         sectors (command ‘u‘).

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-261, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-261, default 261):
Using default value 261

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

[root@Centos-HA2 ~]#mkdir /data

(4).时间同步

# ntpdate -u asia.pool.ntp.org

2、DRBD的安装配置

(1).安装依赖包：( 节点1和节点2)

#yum install gcc gcc-c++ make glibc flex kernel kernel-devel kernel-headers

(2)安装DRBD: (节点1和节点2)

# wget http://oss.linbit.com/drbd/8.4/drbd-8.4.3.tar.gz

# tar -zxvf drbd-8.4.3.tar.gz

# cd drbd-8.4.3

# ./configure --prefix=/usr/local/drbd --with-km

# make KDIR=/usr/src/kernels/2.6.32-431.29.2.el6.x86_64/

# make install

# mkdir -p /usr/local/drbd/var/run/drbd

# cp /usr/local/drbd/etc/rc.d/init.d/drbd /etc/init.d/

# chkconfig --add drbd

# chkconfig --level 35 drbd on

(3)加载DRBD模块(节点1和节点2)

#modprobe drbd

如果出现如下错误

[root@Centos-HA1 drbd-8.4.3]# modprobe drbd
FATAL: Module drbd not found.

解决办法：

http://88fly.blog.163.com/blog/static/1226803902013111345347479

(注意：不要使用yum install kernel* 因为我做实验然后重启系统进不去了

可以使用yum install kernel kernel-devel kernel-headers)

查看DRBD模块是否加载到内核： (节点1和节点2)

# lsmod |grep drbd

节点1

节点2

3.参数配置：(节点1和节点2)

# vi /usr/local/drbd/etc/drbd.conf

清空文件内容，并添加如下配置：

---------------

resource r0{

protocol C;

startup { wfc-timeout 0; degr-wfc-timeout 120;}

disk { on-io-error detach;}

net{

timeout 60;

connect-int 10;

ping-int 10;

max-buffers 2048;

max-epoch-size 2048;

}

syncer { rate 30M;}

on Centos-HA1{

device /dev/drbd0;

disk /dev/sdb1;

address 192.168.1.202:7788;

meta-disk internal;

}

on Centos-HA2{

device /dev/drbd0;

disk /dev/sdb1;

address 192.168.1.203:7788;

meta-disk internal;

}

---------------

(注:至于参数详解请参考 http://www.cnblogs.com/Elliot-wang/p/3994362.html )

(4).创建DRBD设备并激活ro资源：(节点1和节点2)

# mknod /dev/drbd0 b 147 0

# drbdadm create-md r0

等待片刻，显示success表示drbd块创建成功

----------------

Writing meta data...

initializing activity log

NOT initializing bitmap

New drbd meta data block successfully created.

--== Creating metadata ==--

As with nodes, we count the total number of devices mirrored by DRBD

at http://usage.drbd.org.

The counter works anonymously. It creates a random number to identify

the device and sends that random number, along with the kernel and

DRBD version, to usage.drbd.org.

http://usage.drbd.org/cgi-bin/insert_usage.pl?

nu=716310175600466686&ru=15741444353112217792&rs=1085704704

* If you wish to opt out entirely, simply enter ‘no‘.

* To continue, just press [RETURN]

success

----------------

再次输入该命令：

# drbdadm create-md r0

成功激活r0

----------------

[need to type ‘yes‘ to confirm] yes

Writing meta data...

initializing activity log

NOT initializing bitmap

New drbd meta data block successfully created.

----------------

5.启动DRBD服务：(节点1,节点2)

# service drbd start

6.查看状态：(node1,node2)

#service drbd status

# cat /proc/drbd

(7).将Centos-HA1主机配置为主节点：(节点1)

[root@Centos-HA1 ~]# drbdsetup /dev/drbd0 primary --force

查看主机状态

节点1

[root@Centos-HA1 ~]#service drbd status

节点2

[root@Centos-HA1 ~]#service drbd status

ro在主从服务器上分别显示 Primary/Secondary和Secondary/Primary

ds显示UpToDate/UpToDate

表示主从配置成功。

(8).挂载DRBD：(节点1)

从刚才的状态上看到mounted和fstype参数为空，所以我们这步开始挂载DRBD到系统目录

[root@Centos-HA1 ~]# mkfs.ext4 /dev/drbd0

[root@Centos-HA1 ~]# mount /dev/drbd0 /data

注：Secondary节点上不允许对DRBD设备进行任何操作，包括只读，所有的读写操作只能在Primary节点上进行，只有当Primary节点挂掉时，Secondary节点才能提升为Primary节点继续工作。

(9).模拟DRBD1故障，DRBD2接管并提升为Primary

(节点1)

[root@Centos-HA1 ~]#cd /data

[root@Centos-HA1 ~]# touch a.txt b.txt c.txt d.txt e.txt

[root@Centos-HA1 ~]#cd ..

[root@Centos-HA1 ~]# umount /data

[root@Centos-HA1 ~]# drbdsetup /dev/drbd0 secondary

注：这里实际生产环境若DRBD1宕机，在DRBD2状态信息中ro的值会显示为Secondary/Unknown,只需要进行DRBD提权操作即可。

(节点2)

[root@Centos-HA2 ~]# drbdsetup /dev/drbd0 primary

[root@Centos-HA2 ~]# mount /dev/drbd0 /data

[root@Centos-HA2 ~]# cd /data

[root@Centos-HA2 ~]# touch f.txt g.txt h.txt j.txt k.txt

[root@Centos-HA2 ~]# ls

--------------

a.txt b.txt c.txt d.txt e.txt f.txt g.txt h.txt j.txt k.txt lost+found

--------------

到此DRBD已经安装完成。

不过如何保证DRBD主从结构的智能切换，实现高可用，这里就需要Heartbeat来实现了。

Heartbeat会在DRBD主端挂掉的情况下，自动切换从端为主端并自动挂载/data分区。

注：（摘自酒哥的构建高可用LINUX服务器第2版）

假设你把Primary的eth0挡掉，然后直接在Secondary上进行主Primary主机的提升，并且mount上，你可能会发现在Primary上测试考入的文件确实同步过来了，之后把Primary的eth0恢复后，看看有没有自动恢复主从关系。经过查看，发现DRBD检测出了Split-Brain的状况，也就是两个节点都处于standalone状态，故障描述如下：Split-Brain detected,dropping connection! 这就是传说中的“脑裂”。

下面本人做个测试

首先将刚才的主备环境切换回来，此步我不再叙说，安照第9步反过来操作即可

下面我来模拟测试

节点1

[root@Centos-HA1 ~]# cd /data
[root@Centos-HA1 ~]# ls

a.txt b.txt c.txt d.txt e.txt f.txt g.txt h.txt j.txt k.txt lost+found

[root@Centos-HA1 ~]# touch aa.txt bb.txt cc.txt

[root@Centos-HA1 ~]# cd ..

[root@Centos-HA1 ~]# umount data

[root@Centos-HA1 ~]# ifdown eth0

节点2

[root@Centos-HA2 ~]#drbdsetup /dev/drbd0 primary

[root@Centos-HA2 ~]# cd /data

[root@Centos-HA2 ~]#mount /dev/drbd0 /data

[root@Centos-HA2 ~]#cd /data

[root@Centos-HA2 ~]#ls

aa.txt bb.txt cc.txt d.txt f.txt h.txt k.txt
a.txt b.txt c.txt e.txt g.txt j.txt lost+found

节点1

[root@Centos-HA1 ~]# ifup eth0

[root@Centos-HA1 ~]#drbdadm secondary r0

[root@Centos-HA1 ~]#service drbd status

节点2

[root@Centos-HA2 ~]# cd /data

[root@Centos-HA2 ~]# touch aaa.txt bbb.txt ccc.txt

[root@Centos-HA2 ~]#ls

aaa.txt a.txt bb.txt ccc.txt c.txt e.txt g.txt j.txt lost+found
aa.txt bbb.txt b.txt cc.txt d.txt f.txt h.txt k.txt

[root@Centos-HA2 ~]# cd ..

[root@Centos-HA2 ~]# umount /data

[root@Centos-HA2 ~]# drbdadm secondary r0

节点1

[root@Centos-HA1 ~]# drbdsetup /dev/drbd0 primary

[root@Centos-HA1 ~]#mount /dev/drbd0 /data

[root@Centos-HA1 ~]# cd /data/
[root@Centos-HA1 ~]# ls
aa.txt bb.txt cc.txt d.txt f.txt h.txt k.txt
a.txt b.txt c.txt e.txt g.txt j.txt lost+found

(注:文件没有同步过来)

解决办法

(先恢复系统到开始前的状态，就是 eth0 down 之后)

节点1

[root@Centos-HA1 ~]# umount /data
[root@Centos-HA1 ~]# drbdadm secondary r0

节点2

[root@Centos-HA2 ~]# drbdsetup /dev/drbd0 primary

[root@Centos-HA2 ~]# mount /dev/drbd0 /data

[root@Centos-HA2 ~]# cd /data
[root@Centos-HA2 ~]# ls
aaa.txt a.txt bb.txt ccc.txt c.txt e.txt g.txt j.txt lost+found
aa.txt bbb.txt b.txt cc.txt d.txt f.txt h.txt k.txt

已经恢复到故障开始的状态，下面开始解决

节点1

[root@Centos-HA1 ~]# drbdadm secondary r0

[root@Centos-HA1 ~]# drbdadm disconnect all

[root@Centos-HA1 ~]# drbdadm --discard-my-data connect r0

切换回来

[root@Centos-HA2 ~]# drbdadm secondary r0

[root@Centos-HA1 ~]# drbdsetup /dev/drbd0 primary

[root@Centos-HA1 ~]# mount /dev/drbd0 /data

[root@Centos-HA1 ~]# cd /data
[root@Centos-HA1 ~]# ls
aaa.txt a.txt bb.txt ccc.txt c.txt e.txt g.txt j.txt lost+found
aa.txt bbb.txt b.txt cc.txt d.txt f.txt h.txt k.txt

(注:本次eth0 宕机实验，如果/data 目录不卸载的话直接 ifdown eth0是不能成功的)

完成

linux下安装drbd详解

标签：des style blog http color io os 使用 ar

原文地址：http://www.cnblogs.com/Elliot-wang/p/3994400.html

踩

(0)

评论一句话评论（0）

分享档案

更多>

2021年07月29日 (22)
2021年07月28日 (40)
2021年07月27日 (32)
2021年07月26日 (79)
2021年07月23日 (29)
2021年07月22日 (30)
2021年07月21日 (42)
2021年07月20日 (16)
2021年07月19日 (90)
2021年07月16日 (35)

周排行