Redis的集群(搭建)

标签：兴趣   响应   需要   b2b   after   tar   images   process   elf   

Redis Cluster采用虚拟槽分区（引入虚拟槽改进的一致性哈希算法），所有的键根据哈希函数映射到0~16383整数槽内，计算公式：slot=CRC16(key)&16383，找到槽，再找到槽所在的节点。槽是集群内数据管理和迁移的基本单位。

Redis Cluster搭建需要3个步骤

1. 准备节点

Redis Cluster节点数量至少6个才能保证组成完整高可用的集群，每个节点需开启集群模式，配置如下：

# cluster

cluster-enabled yes

cluster-node-timeout 15000

cluster-config-file "nodes-${port}.conf"

cluster-slave-validity-factor 10

cluster-migration-barrier 1

cluster-require-full-coverage yes

启动6个节点，集群模式的Redis除了原有的配置文件redis.conf，又加了一份集群配置文件nodes-${port}.conf，该文件记录了集群内节点信息的变化，如添加/下线节点，故障转移等，由Redis自动维护。

~/6879/data $ cat nodes-6879.conf 

51a3c0a30f397cf28d9f36330cb21df1edda25af :0 myself,master - 0 0 0 connected

vars currentEpoch 0 lastVoteEpoch 0

127.0.0.1:6879> info cluster

# Cluster

cluster_enabled:1

127.0.0.1:6879> cluster info

cluster_state:fail

cluster_slots_assigned:0

cluster_slots_ok:0

cluster_slots_pfail:0

cluster_slots_fail:0

cluster_known_nodes:1

cluster_size:0

cluster_current_epoch:0

cluster_my_epoch:0

cluster_stats_messages_sent:0

cluster_stats_messages_received:0

127.0.0.1:6879> cluster nodes

51a3c0a30f397cf28d9f36330cb21df1edda25af :6879 myself,master - 0 0 0 connected

目前每个节点只能识别出自己的信息，并不知道对方的存在，下面通过握手让6个节点彼此建立联系组成一个集群。

2. 节点握手

节点握手是指一批运行在集群模式下的节点通过Gossip协议彼此通信，达到感知对方的过程。在集群内任意节点上执行cluster meet命令加入新节点，握手状态会通过消息在集群内传播，这样其它节点会自动发现新节点并发起握手流程。

127.0.0.1:6879> cluster meet 127.0.0.1 6880

127.0.0.1:6879> cluster meet 127.0.0.1 6881

127.0.0.1:6879> cluster meet 127.0.0.1 6882

127.0.0.1:6879> cluster meet 127.0.0.1 6883

127.0.0.1:6879> cluster meet 127.0.0.1 6884

127.0.0.1:6879> cluster nodes

51a3c0a30f397cf28d9f36330cb21df1edda25af 127.0.0.1:6879 myself,master - 0 0 1 connected

aa94bcbe1f7ebcf850e1d75cad712a0bbc044d97 127.0.0.1:6884 master - 0 1532681718583 5 connected

8c0e31b4cadc12c784eaa63a200fbb9b86e49a72 127.0.0.1:6880 master - 0 1532681713007 2 connected

ae5c6df67610f399491c174a6ee97345e03fd610 127.0.0.1:6882 master - 0 1532681716555 3 connected

b298f797f42b2fe1269f83e9fadd6cbb2af2fd04 127.0.0.1:6881 master - 0 1532681717567 4 connected

ff0092bfdd407ea73f036564e48b277898494ac1 127.0.0.1:6883 master - 0 1532681719596 0 connected

节点握手后还不能正常工作，这时集群处于下线状态，是由于槽没有分配到节点，集群无法完成槽到节点的映射。

127.0.0.1:6879> set hello redis

(error) CLUSTERDOWN Hash slot not served

3. 分配槽

Redis cluster把所有数据映射到16384个槽中，每个key会映射为一个固定的槽，只有当节点分配了槽，才能响应和这些槽关联的键命令，下面通过cluster addslots命令为节点分配槽。

$ redis-cli -p 6879 cluster addslots {0..5641}                             

$ redis-cli -p 6880 cluster addslots {5642..10922}  

$ redis-cli -p 6881 cluster addslots {10923..16383} 

当前集群状态是ok，进入在线状态，所有槽都已经分配给节点，cluster nodes命令可看到槽和节点的对应关系。

127.0.0.1:6879> cluster info

cluster_state:ok

cluster_slots_assigned:16384

cluster_slots_ok:16384

cluster_slots_pfail:0

cluster_slots_fail:0

cluster_known_nodes:6

cluster_size:3

cluster_current_epoch:5

cluster_my_epoch:1

cluster_stats_messages_sent:2864

cluster_stats_messages_received:2864

127.0.0.1:6879> cluster nodes

51a3c0a30f397cf28d9f36330cb21df1edda25af 127.0.0.1:6879 myself,master - 0 0 1 connected 0-5641

aa94bcbe1f7ebcf850e1d75cad712a0bbc044d97 127.0.0.1:6884 master - 0 1532683047361 5 connected

8c0e31b4cadc12c784eaa63a200fbb9b86e49a72 127.0.0.1:6880 master - 0 1532683048374 2 connected 5642-10922

ae5c6df67610f399491c174a6ee97345e03fd610 127.0.0.1:6882 master - 0 1532683046350 3 connected

b298f797f42b2fe1269f83e9fadd6cbb2af2fd04 127.0.0.1:6881 master - 0 1532683044329 4 connected 10923-16383

ff0092bfdd407ea73f036564e48b277898494ac1 127.0.0.1:6883 master - 0 1532683045341 0 connected

目前还有3个节点没有使用，作为一个完整的集群，每个负责处理槽的节点应该具有从节点，保证出现故障时，可以进行自动故障转移，使用cluster replicate命令让一个节点成为从节点。

127.0.0.1:6882> cluster replicate 51a3c0a30f397cf28d9f36330cb21df1edda25af

127.0.0.1:6883> cluster replicate 8c0e31b4cadc12c784eaa63a200fbb9b86e49a72

127.0.0.1:6884> cluster replicate b298f797f42b2fe1269f83e9fadd6cbb2af2fd04

127.0.0.1:6884> cluster nodes

8c0e31b4cadc12c784eaa63a200fbb9b86e49a72 127.0.0.1:6880 master - 0 1532683770612 2 connected 5642-10922

ae5c6df67610f399491c174a6ee97345e03fd610 127.0.0.1:6882 slave 51a3c0a30f397cf28d9f36330cb21df1edda25af 0 1532683773646 3 connected

b298f797f42b2fe1269f83e9fadd6cbb2af2fd04 127.0.0.1:6881 master - 0 1532683772637 4 connected 10923-16383

ff0092bfdd407ea73f036564e48b277898494ac1 127.0.0.1:6883 slave 8c0e31b4cadc12c784eaa63a200fbb9b86e49a72 0 1532683774150 2 connected

51a3c0a30f397cf28d9f36330cb21df1edda25af 127.0.0.1:6879 master - 0 1532683769602 1 connected 0-5641

aa94bcbe1f7ebcf850e1d75cad712a0bbc044d97 127.0.0.1:6884 myself,slave b298f797f42b2fe1269f83e9fadd6cbb2af2fd04 0 0 5 connected

目前为止，手动建立了一个6个节点的集群，3个主节点负责处理槽和相关数据，3个从节点负责故障转移。

手动搭建集群，步骤比较繁琐，Redis官方提供了redis-trib.rb工具方便快速搭建，该工具采用Ruby实现，使用需先准备Ruby环境。

准备Ruby环境

# wget https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.1.tar.gz

# tar zxf ruby-2.3.1.tar.gz

# ./configure --prefix=/usr/local/ruby

# make

# make install

# wget http://rubygems.org/downloads/redis-3.3.0.gem

# gem install --local redis-3.3.0.gem 

Successfully installed redis-3.3.0

Parsing documentation for redis-3.3.0

Installing ri documentation for redis-3.3.0

Done installing documentation for redis after 0 seconds

1 gem installed

安装Ruby环境后，执行redis-trib.rb命令确认环境是否正确。

$ redis-trib.rb 

Usage: redis-trib <command> <options> <arguments ...>

...

创建集群

$ redis-trib.rb create --replicas 1 127.0.0.1:6879 127.0.0.1:6880 127.0.0.1:6881 127.0.0.1:6882 127.0.0.1:6883 127.0.0.1:6884      

>>> Creating cluster

>>> Performing hash slots allocation on 6 nodes...

Using 3 masters:

127.0.0.1:6879

127.0.0.1:6880

127.0.0.1:6881

Adding replica 127.0.0.1:6882 to 127.0.0.1:6879

Adding replica 127.0.0.1:6883 to 127.0.0.1:6880

Adding replica 127.0.0.1:6884 to 127.0.0.1:6881

M: 93aec643effa795f33ab2b151c6a2b273eeb5462 127.0.0.1:6879

   slots:0-5460 (5461 slots) master

M: 22b6e422aed074e42d295c061f0b4c102304b5bb 127.0.0.1:6880

   slots:5461-10922 (5462 slots) master

M: 1835a6b9b3e0c0c08b09a6798c0359996be7ba7b 127.0.0.1:6881

   slots:10923-16383 (5461 slots) master

S: 3ae8e6622d37e15701a757139a51c0006a6df664 127.0.0.1:6882

   replicates 93aec643effa795f33ab2b151c6a2b273eeb5462

S: 324228642e6fc87a0c367a6b8e7a47e2879aa7d5 127.0.0.1:6883

   replicates 22b6e422aed074e42d295c061f0b4c102304b5bb

S: 178525484b9e865522a0cfa2fef7f207413272d3 127.0.0.1:6884

   replicates 1835a6b9b3e0c0c08b09a6798c0359996be7ba7b

Can I set the above configuration? (type 'yes' to accept): yes

>>> Nodes configuration updated

>>> Assign a different config epoch to each node

>>> Sending CLUSTER MEET messages to join the cluster

Waiting for the cluster to join...

>>> Performing Cluster Check (using node 127.0.0.1:6879)

M: 93aec643effa795f33ab2b151c6a2b273eeb5462 127.0.0.1:6879

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

M: 22b6e422aed074e42d295c061f0b4c102304b5bb 127.0.0.1:6880

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

S: 3ae8e6622d37e15701a757139a51c0006a6df664 127.0.0.1:6882

   slots: (0 slots) slave

   replicates 93aec643effa795f33ab2b151c6a2b273eeb5462

M: 1835a6b9b3e0c0c08b09a6798c0359996be7ba7b 127.0.0.1:6881

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

S: 324228642e6fc87a0c367a6b8e7a47e2879aa7d5 127.0.0.1:6883

   slots: (0 slots) slave

   replicates 22b6e422aed074e42d295c061f0b4c102304b5bb

S: 178525484b9e865522a0cfa2fef7f207413272d3 127.0.0.1:6884

   slots: (0 slots) slave

   replicates 1835a6b9b3e0c0c08b09a6798c0359996be7ba7b

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

查看集群信息

$ redis-trib.rb info 127.0.0.1:6879

127.0.0.1:6879 (93aec643...) -> 0 keys | 5461 slots | 1 slaves.

127.0.0.1:6880 (22b6e422...) -> 0 keys | 5462 slots | 1 slaves.

127.0.0.1:6881 (1835a6b9...) -> 0 keys | 5461 slots | 1 slaves.

[OK] 0 keys in 3 masters.

0.00 keys per slot on average.

检查集群完整性

redis-trib.rb check 127.0.0.1:6879

>>> Performing Cluster Check (using node 127.0.0.1:6879)

M: 93aec643effa795f33ab2b151c6a2b273eeb5462 127.0.0.1:6879

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

M: 22b6e422aed074e42d295c061f0b4c102304b5bb 127.0.0.1:6880

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

S: 3ae8e6622d37e15701a757139a51c0006a6df664 127.0.0.1:6882

   slots: (0 slots) slave

   replicates 93aec643effa795f33ab2b151c6a2b273eeb5462

M: 1835a6b9b3e0c0c08b09a6798c0359996be7ba7b 127.0.0.1:6881

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

S: 324228642e6fc87a0c367a6b8e7a47e2879aa7d5 127.0.0.1:6883

   slots: (0 slots) slave

   replicates 22b6e422aed074e42d295c061f0b4c102304b5bb

S: 178525484b9e865522a0cfa2fef7f207413272d3 127.0.0.1:6884

   slots: (0 slots) slave

   replicates 1835a6b9b3e0c0c08b09a6798c0359996be7ba7b

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

若感兴趣可关注订阅号”数据库最佳实践”(DBBestPractice).

Redis的集群(搭建)

标签：兴趣   响应   需要   b2b   after   tar   images   process   elf   

原文地址：http://blog.51cto.com/coveringindex/2151249