标签：pass usr efi user clu mongod one 集群 --
前提条件：
mongo集群必须是副本集，因为同步插件读取的是日志文件
1.需要安装pip插件（如果已经安装不需再安装）
yum install python-setuptools && easy_install pip
2.在mongo端 安装 mongo-connector。
pip install mongo-connector
3.安装 elastic2-doc-manager（注意这个是针对es2.x版本的，如果es安装的是1.x版本，用elastic-doc-manager）
pip install elastic-doc-manager
插件网址：https://github.com/mongodb-labs
4.启动同步插件
2.使用配置文件
| { "__comment__": "Configuration options starting with ‘__‘ are disabled", "__comment__": "To enable them, remove the preceding ‘__‘", "mainAddress": "192.168.106.220:27017", "oplogFile": "/home/mongo/oplog.timestamp", 自定义 "noDump": false, "batchSize": -1, "verbosity": 0, "__continueOnError": true,当数据中有错误会跳过 "logging": { "type": "file", "filename": "/home/mongo/mongo-connector.log", 自定义 "__format": "%(asctime)s [%(levelname)s] %(name)s:%(lineno)d - %(message)s", "__rotationWhen": "D", "__rotationInterval": 1, "__rotationBackups": 10, "__type": "syslog", "__host": "localhost:514" }, "authentication": { "__adminUsername": "username", "__password": "password", "__passwordFile": "mongo-connector.pwd" }, "__comment__": "For more information about SSL with MongoDB, please see http://docs.mongodb.org/manual/tutorial/configure-ssl-clients/", "__ssl": { "__sslCertfile": "Path to certificate to identify the local connection against MongoDB", "__sslKeyfile": "Path to the private key for sslCertfile. Not necessary if already included in sslCertfile.", "__sslCACerts": "Path to concatenated set of certificate authority certificates to validate the other side of the connection", "__sslCertificatePolicy": "Policy for validating SSL certificates provided from the other end of the connection. Possible values are ‘required‘ (require and validate certificates), ‘optional‘ (validate but don‘t require a certificate), and ‘ignored‘ (ignore certificates)." }, "__fields": ["field1", "field2", "field3"], "__namespaces": { "excluded.collection": false, "excluded_wildcard.*": false, "*.exclude_collection_from_every_database": false, "included.collection1": true, "included.collection2": {}, "included.collection4": { "includeFields": ["included_field", "included.nested.field"] }, "included.collection5": { "rename": "included.new_collection5_name", "includeFields": ["included_field", "included.nested.field"] }, "included.collection6": { "excludeFields": ["excluded_field", "excluded.nested.field"] }, "included.collection7": { "rename": "included.new_collection7_name", "excludeFields": ["excluded_field", "excluded.nested.field"] }, "included_wildcard1.*": true, "included_wildcard2.*": true, "renamed.collection1": "something.else1", "renamed.collection2": { "rename": "something.else2" }, "renamed_wildcard.*": { "rename": "new_name.*" }, "gridfs.collection": { "gridfs": true }, "gridfs_wildcard.*": { "gridfs": true } }, "docManagers": [ { "docManager": "elastic_doc_manager", "targetURL": ["192.168.106.218:9200","192.168.106.217:9200","192.168.106.219:9200"], "bulkSize": 1000, "__uniqueKey": "_id", "autoCommitInterval": 1 } ] } |
启动命令：mongo-connector -c /etc/mongo-connector-conf.json & （json的目录自定义）
注意：写入mongo的速度比同步速度快，经过测试elasticsearch配置良好的情况下同步速度大概在1000条/s左右
插件安装的目录在：/usr/lib/python2.7/site-packages下面
elasticsearch、mongo-connector（如果想修改源码可以进入此目录）
标签：pass usr efi user clu mongod one 集群 --
原文地址：http://www.cnblogs.com/yhx-0408/p/7267704.html
