标签：
为了提升逼格，我决定使用英文版的的这句作为开头，尽管我最初想到的是中文版orz
天灾人祸这种事，总归是有的。尽管服务器的灾难性故障不常发生，概率也比遇到鬼或是遇到爱情来得高些，将风险考虑在内总归是好的，所谓有备无患。如果你已经在自己的架构中将单点故障考虑进去了，就不必往下拉啦，继续舔酸奶盖去吧~如果你的云服务商已经考虑了单点故障，你也没啥好做啦。
如果两者都没有，你应该请我喝杯星冰乐再往下拉。
ps：我最近多是在处理性能优化，数据迁移，安全策略，系统稳定性这类事，简直是伪装成开发的运维orz
采用异地备份来应对单点故障导致的数据丢失
首先明确我们需要备份哪些数据,关于edX数据，可以参考我此前的这篇文章
一番深(hu)思(si)熟(luan)虑(xiang)之后，你会发现，我们关心的数据无非是数据库(mongodb/mysql)里存的，以及静态文件(/edx/var/edxapp/staticfiles)
思路很简单，每天定时备份数据库和静态文件，将其同步到远程服务器。只保存最近3，5天的备份文件，逾期的删除
这里需要注意的问题可能有三点：
那么写个shell脚本基本就完事啦，主要可能涉及的工具：
就在我shell脚本快写完的时候，在github上发现了这个东西：backup
一言话概括就是
Easy full stack backup operations on UNIX-like systems
稍微扫一眼Overview,分分钟爱不释手啊。
从文档中不难发现，backup本质上是对以上我们提到的工具做了封装。
能不写shell，我是尽量不写shell的。
先读下文档吧。
backup是ruby写的，使用gem安装。不得不说ruby的DSL能力真强啊
sudo gem install backup
哦，顺便说下，我在ruby1.8.7和ruby1.9.3下都没成功，之后在ubuntu12.04上手动编译了ruby2.0.0才安装好backup。也是蛮诡异的，可能是我机器的问题，文档里说ruby1.8.7和ruby1.9.3应该都是没有问题的
从一个简单的案例开始。
任务描述：备份我的home目录下的所有文件，同步到远程服务器的~/backup_my_home目录，只保留最近的5个备份
在shell里执行
backup generate:model --trigger backup_my_home --archives --storages=‘scp‘ --compressor=‘gzip‘
会生成:Generated model file: ‘/home/USERNAME/Backup/models/backup_my_home.rb‘
编辑以上文件就行
# encoding: utf-8
Model.new(:backup_my_home, ‘Description for backup_my_home‘) do
archive :my_archive do |archive|
# Run the `tar` command using `sudo`
# archive.use_sudo
archive.add "/home/USERNAME"
end
# SCP (Secure Copy) [Storage]
#
store_with SCP do |server|
server.username = "my_username"
server.password = "my_password"
server.ip = "123.45.678.90"
server.port = 22
server.path = "~/backup_my_home/"
server.keep = 5
end
# Gzip [Compressor]
compress_with Gzip
end
几乎无需解释对吧:)
开始备份:backup perform -t backup_my_home
backup generate:model --trigger edx_staticfile_backup --archives --storages=‘scp‘ --compressor=‘gzip‘
# encoding: utf-8
Model.new(:edx_staticfile_backup, ‘Description for edx_staticfile_backup‘) do
# archive.root ‘/path/to/archive/root‘
#
archive :my_archive do |archive|
# Run the `tar` command using `sudo`
# archive.use_sudo
#archive.add "/path/to/a/file.rb"
archive.add "/edx/var/edxapp/staticfiles"
#archive.exclude "/path/to/a/excluded_file.rb"
#archive.exclude "/path/to/a/excluded_folder"
end
##
# SCP (Secure Copy) [Storage]
#你需要先去目标服务器建立~/edx_backups/staticfile目录
store_with SCP do |server|
server.username = "my_username"
server.password = "my_password"
server.ip = "123.45.678.90"
server.port = 22
server.path = "~/edx_backups/staticfile"
server.keep = 5
# Additional options for the SSH connection.
# server.ssh_options = {}
end
##
# Gzip [Compressor]
#
compress_with Gzip
end
>mongo
>show dbs
admin
cs_comments_service_development
edxapp
local
>mongo
>use edxapp
>db.addUser(“wwj”, “xxx”)
backup generate:model --trigger edx_mongodb_backup --databases="mongodb" --storages=‘scp‘ --compressor=‘gzip‘ --notifiers=‘mail‘
# encoding: utf-8
Model.new(:edx_mongodb_backup, ‘Description for edx_mongodb_backup‘) do
##
# MongoDB [Database]
#
database MongoDB do |db|
db.name = "edxapp"
#db.name = :all
db.username = "wwj"
db.password = "xxx"
db.host = "localhost"
db.port = 27017
db.ipv6 = false
#db.only_collections = ["only", "these", "collections"]
#db.additional_options = []
db.lock = false
db.oplog = false
end
##
# SCP (Secure Copy) [Storage]
#
store_with SCP do |server|
server.username = "my_username"
server.password = "my_password"
server.ip = "123.45.678.90"
server.port = 22
server.path = "~/edx_backups/mongo"
server.keep = 5
# Additional options for the SSH connection.
# server.ssh_options = {}
end
##
# Gzip [Compressor]
#
compress_with Gzip
notify_by Mail do |mail|
mail.on_success = true
mail.on_warning = true
mail.on_failure = true
mail.from = "sender@email.com"
mail.to = "receiver@email.com"
mail.address = "smtp.gmail.com"
mail.port = 587
mail.domain = "your.host.name"
mail.user_name = "sender@email.com"
mail.password = "my_password"
mail.authentication = "plain"
mail.encryption = :starttls
end
end
text
>mysql -u root
>show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| edxapp |
| mysql |
| ora |
| performance_schema |
| test |
| xqueue |
+--------------------+
backup generate:model --trigger edx_mysql_backup --databases="mysql" --storages=‘scp‘ --compressor=‘gzip‘
# encoding: utf-8
Model.new(:edx_mysql_backup, ‘Description for edx_mysql_backup‘) do
##
# MySQL [Database]
#
database MySQL do |db|
# To dump all databases, set `db.name = :all` (or leave blank)
#db.name = "my_database_name"
db.name = "edxapp"
db.username = "root"
db.password = ""
db.host = "localhost"
db.port = 3306
#mysql.sock 位置有变
#db.socket = "/tmp/mysql.sock"
# Note: when using `skip_tables` with the `db.name = :all` option,
# table names should be prefixed with a database name.
# e.g. ["db_name.table_to_skip", ...]
#db.skip_tables = ["skip", "these", "tables"]
#db.only_tables = ["only", "these", "tables"]
db.additional_options = ["--quick", "--single-transaction"]
end
##
# SCP (Secure Copy) [Storage]
#
store_with SCP do |server|
server.username = "my_username"
server.password = "my_password"
server.ip = "123.45.678.90"
server.port = 22
server.path = "~/edx_backups/mysql"
server.keep = 5
# Additional options for the SSH connection.
# server.ssh_options = {}
end
##
# Gzip [Compressor]
#
compress_with Gzip
end
backup perform -t edx_staticfile_backup
backup perform -t edx_mongodb_backup
backup perform -t edx_mysql_backup
每天凌晨3:00
sudo crontab -e写入
00 03 * * * /usr/local/bin/backup perform -t edx_staticfile_backup
00 03 * * * /usr/local/bin/backup perform -t edx_mongodb_backup
00 03 * * * /usr/local/bin/backup perform -t edx_mysql_backup
之后我们还是采用rsync的做法吧，不然数据大到一定程度，太浪费带宽了
只要更改store_with就行
注意：数据服务器到备份服务器应当配置好ssh免登陆
compress_with Gzip do |gzip|
gzip.rsyncable = true
end
store_with RSync do |storage|
time = Time.now
storage.mode = :ssh
storage.host = "123.45.67.89"
storage.port = 22
storage.ssh_user = "wwj"
#免密码ssh
#每周为一轮，使用的是同名覆盖
path = time.strftime ‘%A‘
storage.path = "~/edx_backups/staticfile/#{ path }"
end
~/Backup/log/backup.log 中的时间是ruby的时间，与本地差（晚）8个小时。时区的问题
标签：
原文地址：http://www.cnblogs.com/zhaojianwei/p/4666881.html
