MongoDB分片群集

标签：primary   and   sys   mongod   table   实验   ffffff   文件储存   通过   

什么是分片

高数据量和吞吐量的数据库应用会对单机的性能造成较大的压力，大的查询量会把单机CPU耗尽，大的数据量对单机的储存压力大，最终会消耗系统的内存而将压力转移到磁盘IO上

MongoDB分片的优势

分片为应对高吞吐量与大数据量提供了方法

• 使用分片减少了对需要处理的请求，因此，通过水平拓展，群集可以提高自己的储存量和吞吐量。
• 使用分片减少了每个分片存储的数据

分片服务器分为以下几个服务器组成

• 路由服务器 （router）

• 配置服务器 （configsvr）

• 分片服务器 （shardsvr）

我们通过路由服务器发送请求给配置服务器，配置服务器调用下面的分片服务器

结构图如下

MongoDB分片群集的部署

====（注：分片群集在3.2和3.2以上的版本不同3.2版本在做分片服务器的时候可以一个分片服务器对应一台数据库，但是在3.2以上的版本它要求每个分片服务器必须做群集）====

这边我们用的是3.2版本的源代码包

实验步骤：做配置服务器

---------------------安装MongoDB 3.2----------------------------
#安装开发包
yum install openssl-devel -y
#解压源代码包
tar zxvf mongodb-linux-x86_64-3.2.1.tgz -C /opt/
#重命名并移动到/usr/local文件里面
mv mongodb-linux-x86_64-3.2.1/ /usr/local/mongodb
#创建数据存放位置
mkdir -p /data/mongodb/mongodb{1,2,3,4}
#创建日志存放位置
mkdir /data/mongodb/logs
创建以log为结尾的日志存放
touch /data/mongodb/logs/mongodb{1,2,3,4}.log
#给日志文件最大全权限
chmod -R 777 /data/mongodb/logs/*.log
#修改最大并发连接
ulimit -n 25000
ulimit -u 25000

上面是开始安装MongoDB，下面开始配置，配置服务器的配置文件

cd /usr/local/mongodb/bin/
vim mongodb1.conf
#配置服务器端口号
port=37017       
#数据文件储存位置
dbpath=/data/mongodb/mongodb1
#日志文件储存位置
logpath=/data/mongodb/logs/mongodb1.log

logappend=true

fork=true
#同时承受的并发连接数
maxConns=5000
storageEngine=mmapv1
#配置为配置服务器
configsvr=true

某节点内存不足时，从其他节点分配内存

sysctl -w vm.zone_reclaim_mode=0
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag

#建立软连接便于管理
ln -s /usr/local/mongodb/bin/mongo /usr/bin/mongo
ln -s /usr/local/mongodb/bin/mongod /usr/bin/mongod
#开启配置服务器
mongod -f /usr/local/mongodb/bin/mongodb1.conf

分片服务器 1和分片服务器 2同样配置改端口号数据文件日志文件位置加入shardsvr=true删除原有的配置服务器configsvr=true

#端口号改为47017
port=37017        
#数据文件储存位置
dbpath=/data/mongodb/mongodb2
#日志文件储存位置
logpath=/data/mongodb/logs/mongodb.log

logappend=true

fork=true
#同时承受的并发连接数
maxConns=5000
storageEngine=mmapv1
#配置为分片服务器
shardsvr=true
#分别开启分片服务器1和2

启动路由服务器

cd /usr/local/mongodb/bin/
#启动mongos 路由服务器的端口号为27017日志存放目录在/usr/local/mongodb/bin/下面的route.log里面
配置服务IP：端口号 --chunkSize 1 块大小为1
./mongos --port 27017 --fork --logpath=/usr/local/mongodb/bin/route.log --configdb 192.168.235.204:37017 --chunkSize 1

直接启动mongo路由服务器

[root@localhost bin]# mongo
MongoDB shell version: 3.2.1
connecting to: test
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
    http://docs.mongodb.org/
Questions? Try the support group
    http://groups.google.com/group/mongodb-user
Server has startup warnings: 
2018-07-18T14:52:06.622+0800 I CONTROL  [main] ** WARNING: You are running this process as the root user, which is not recommended.
2018-07-18T14:52:06.622+0800 I CONTROL  [main] 
mongos>
#查看数据库
mongos> show dbs
config  0.031GB
mongos> sh.status()   #shards下为空，没有分片服务器
--- Sharding Status --- 
  sharding version: {
    "_id" : 1,
    "minCompatibleVersion" : 5,
    "currentVersion" : 6,
    "clusterId" : ObjectId("5b4ee3961c914459f603260f")
}
  shards:
  active mongoses:
    "3.2.1" : 1
  balancer:
    Currently enabled:  yes
    Currently running:  no
    Failed balancer rounds in last 5 attempts:  0
    Migration Results for the last 24 hours: 
        No recent migrations
  databases:
  #添加分片服务器
  mongos> sh.addShard("192.168.32.207:47018")
{ "shardAdded" : "shard0000", "ok" : 1 }
mongos> sh.addShard("192.168.32.207:47017")
{ "shardAdded" : "shard0001", "ok" : 1 }
mongos> sh.status()
--- Sharding Status --- 
  sharding version: {
    "_id" : 1,
    "minCompatibleVersion" : 5,
    "currentVersion" : 6,
    "clusterId" : ObjectId("5b4ee3961c914459f603260f")
}
  shards:  #下面两个是分片服务器
    {  "_id" : "shard0000",  "host" : "192.168.32.207:47018" }
    {  "_id" : "shard0001",  "host" : "192.168.32.207:47017" }
  active mongoses:
    "3.2.1" : 1
  balancer:
    Currently enabled:  yes
    Currently running:  no
    Failed balancer rounds in last 5 attempts:  0
    Migration Results for the last 24 hours: 
        No recent migrations
  databases:

实验分片功能

添加两个分片服务器后，数据库和集合还未启用分片使用mongoimport命令导入sales.txt数据到kgc数据库的sales表

mongos> db.createCollection(‘users‘)
{ "ok" : 1 }
mongos> show collections
system.indexes
users
mongos> exit
bye
[root@localhost bin]# mongo
MongoDB shell version: 3.2.1
connecting to: test
Server has startup warnings: 
2018-07-18T14:52:06.622+0800 I CONTROL  [main] ** WARNING: You are running this process as the root user, which is not recommended.
2018-07-18T14:52:06.622+0800 I CONTROL  [main] 
mongos> show dbs
config  0.031GB
kgc     0.078GB
mongos> for (var i=1;i<=10000;i++)db.users.insert({"id":1,"name":"jaap"+i}) #插入10000条信息
WriteResult({ "nInserted" : 1 })
mongos> show tables
system.indexes
users
mongos> db.users.find() #查看信息
{ "_id" : ObjectId("5b4ef398b83d741f7c50491a"), "id" : 1, "name" : "zhang" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50491b"), "id" : 1, "name" : "jaap1" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50491c"), "id" : 1, "name" : "jaap2" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50491d"), "id" : 1, "name" : "jaap3" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50491e"), "id" : 1, "name" : "jaap4" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50491f"), "id" : 1, "name" : "jaap5" }
{ "_id" : ObjectId("5b4ef407b83d741f7c504920"), "id" : 1, "name" : "jaap6" }
{ "_id" : ObjectId("5b4ef407b83d741f7c504921"), "id" : 1, "name" : "jaap7" }
{ "_id" : ObjectId("5b4ef407b83d741f7c504922"), "id" : 1, "name" : "jaap8" }
{ "_id" : ObjectId("5b4ef407b83d741f7c504923"), "id" : 1, "name" : "jaap9" }
{ "_id" : ObjectId("5b4ef407b83d741f7c504924"), "id" : 1, "name" : "jaap10" }
{ "_id" : ObjectId("5b4ef407b83d741f7c504925"), "id" : 1, "name" : "jaap11" }
{ "_id" : ObjectId("5b4ef407b83d741f7c504926"), "id" : 1, "name" : "jaap12" }
{ "_id" : ObjectId("5b4ef407b83d741f7c504927"), "id" : 1, "name" : "jaap13" }
{ "_id" : ObjectId("5b4ef407b83d741f7c504928"), "id" : 1, "name" : "jaap14" }
{ "_id" : ObjectId("5b4ef407b83d741f7c504929"), "id" : 1, "name" : "jaap15" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50492a"), "id" : 1, "name" : "jaap16" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50492b"), "id" : 1, "name" : "jaap17" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50492c"), "id" : 1, "name" : "jaap18" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50492d"), "id" : 1, "name" : "jaap19" }
Type "it" for more
mongos> db.users.find().limit(5)  #查看指定信息
{ "_id" : ObjectId("5b4ef398b83d741f7c50491a"), "id" : 1, "name" : "zhang" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50491b"), "id" : 1, "name" : "jaap1" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50491c"), "id" : 1, "name" : "jaap2" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50491d"), "id" : 1, "name" : "jaap3" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50491e"), "id" : 1, "name" : "jaap4" }
mongos> sh.status()  #查看数据库分片信息
--- Sharding Status --- 
  sharding version: {
    "_id" : 1,
    "minCompatibleVersion" : 5,
    "currentVersion" : 6,
    "clusterId" : ObjectId("5b4ee3961c914459f603260f")
}
  shards:
    {  "_id" : "shard0000",  "host" : "192.168.32.207:47018" }
    {  "_id" : "shard0001",  "host" : "192.168.32.207:47017" }
  active mongoses:
    "3.2.1" : 1
  balancer:
    Currently enabled:  yes
    Currently running:  no
    Failed balancer rounds in last 5 attempts:  0
    Migration Results for the last 24 hours: 
        No recent migrations
  databases:
    {  "_id" : "kgc",  "primary" : "shard0000",  "partitioned" : false }

启用分片

mongos> sh.enableSharding("kgc")
{ "ok" : 1 }
mongos> sh.status()
--- Sharding Status --- 
  sharding version: {
    "_id" : 1,
    "minCompatibleVersion" : 5,
    "currentVersion" : 6,
    "clusterId" : ObjectId("5b4ee3961c914459f603260f")
}
  shards:
    {  "_id" : "shard0000",  "host" : "192.168.32.207:47018" }
    {  "_id" : "shard0001",  "host" : "192.168.32.207:47017" }
  active mongoses:
    "3.2.1" : 1
  balancer:
    Currently enabled:  yes
    Currently running:  no
    Failed balancer rounds in last 5 attempts:  0
    Migration Results for the last 24 hours: 
        No recent migrations
  databases:
    {  "_id" : "kgc",  "primary" : "shard0000",  "partitioned" : true

#对users表中id创建主键索引
mongos> db.users.createIndex({"id":1})
#表分片
mongos> sh.shardCollection("kgc.users",{"id":1})
#查看分片情况
mongos> sh.status()
--- Sharding Status --- 
  sharding version: {
    "_id" : 1,
    "minCompatibleVersion" : 5,
    "currentVersion" : 6,
    "clusterId" : ObjectId("5b4ee3961c914459f603260f")
}
  shards:
    {  "_id" : "shard0000",  "host" : "192.168.32.207:47018" }
    {  "_id" : "shard0001",  "host" : "192.168.32.207:47017" }
  active mongoses:
    "3.2.1" : 1
  balancer:
    Currently enabled:  yes
    Currently running:  no
    Failed balancer rounds in last 5 attempts:  0
    Migration Results for the last 24 hours: 
        No recent migrations
  databases:
    {  "_id" : "kgc",  "primary" : "shard0000",  "partitioned" : true }
        kgc.users
            shard key: { "id" : 1 }
            unique: false
            balancing: true
            chunks:
                shard0000   1
            { "id" : { "$minKey" : 1 } } -->> { "id" : { "$maxKey" : 1 } } on : shard0000 Timestamp(1, 0)

添加标签

#添加分片服务器

mongod -f mongodb4.conf
mongo

mongos> sh.addShard("192.168.235.205:47019")

mongos> sh.status()

chunks:
    shard0000   4
    shard0001   4
    shard0002   3

#删除分片节点
mongos> use admin

mongos> db.runCommand({"removeshard":"192.168.235.205:47019"})

MongoDB分片群集

标签：primary   and   sys   mongod   table   实验   ffffff   文件储存   通过   

原文地址：http://blog.51cto.com/13645280/2146831