TimesTen 数据库复制学习：10. 定义classic复制

设计高可用系统

复制的目标为：
1. 提供一个或多个复制数据库，保证数据可以为应用所用
2. 提供复制数据库用于恢复
3. 负载均衡
4. 无中断的软件升级和维护

classic replication scheme支持以下形式：
Unidirectional - 这个和ASP有和区别，会切换角色吗
Bidirectional split workload - 双向，属于互备型，两个数据库负责不同的工作负载
Bidirectional distributed workload - 双向，属于双活型
Propagation - 数据分发型

作为一个高可用系统，需要考虑以下几方面

1. Considering failover and recovery scenarios
   当master数据库失效时，需要自动的将应用重新定向到可用的数据库，这通常是通过集群软件或客户应用逻辑实现的。这些在Chapter 15, “Managing Database Failover and Recovery”. 中讨论
   A bidirectional scheme that replicates the entire database can take advantage of automatic restoration of a failed master. 何出此言：
   对于单向复制，如果master失效，或者等待master从subscriber恢复（应用不需要redirect，但有中断），或者使用ALTER REPLICATION将subscriber定义为master（应用需要redirect，中断较少）。
   双向复制的好处在于，当某一master失效时，对于split workload，应用，应用只需重定向即可（可以使用automatic client failover）；对于distributed workload，无需重定向，但需要处理数据冲突即可。期间可以用ALTER REPLICATION设置新的master

2. Making decisions about performance and recovery tradeoffs
   应尽量简化错误切换和恢复的过程，这与复制的拓扑有关
   和ASP一样，classic复制也支持异步，半同步和全同步复制。全同步对性能影响最大
   Return receipt can be used in more configurations, whereas return twosafe can only be used in a bidirectional configuration or an active standby pair

3. Distributing workloads
   配置双向复制，通常用于读比较多的应用。如果写比较多，则需考虑性能影响并防止更新冲突

定义classic replication scheme

classic和ASP的scheme是不能并存的，如果已经定义了ASP，则创建classic复制时报错：

cachedb1> CREATE REPLICATION repscheme  ELEMENT e TABLE a3                                                              MASTER cachedb1 ON "timesten-hol"                                               SUBSCRIBER cachedb2 ON "timesten-hol"                                           RETURN RECEIPT;
 8126: An ACTIVE STANDBY PAIR replication scheme already exists and must be the only replication scheme for the store.

一个复制scheme包含多个element，定义复制的对象，每个element有DATASTORE, TABLE, 或SEQUENCE三种类型
一个复制的表只能位于一个element中，并且只能指定一个角色
ALTER REPLICATION可以后续在scheme中添加复制对象。
注意, ALTER REPLICATION只针对classic复制。

复制整个库，Defining the DATASTORE element，如
ELEMENT ds1 DATASTORE
MASTER masterds ON “system1”
SUBSCRIBER subscriberds ON “system2”
可以在DATASTORE中仅复制某些对象或指定不复制某些对象，如
EXCLUDE TABLE ttuser.tab1, ttuser.tab2
EXCLUDE SEQUENCE ttuser.seq1
或
INCLUDE TABLE ttuser.tab3
INCLUDE SEQUENCE ttuser.seq2, ttuser.seq3

复制某张表，Defining table elements，如:
ELEMENT a TABLE ttuser.tab1
MASTER masterds ON “system1”
SUBSCRIBER subscriberds ON “system2”
ELEMENT b TABLE ttuser.tab2
MASTER masterds ON “system1”
SUBSCRIBER subscriberds ON “system2”

还可以复制外键和主键关联的一组表

可以复制sequence

不直接支持复制materialized view，但可以通过复制物化视图关联的事实表来间接支持

classic 复制中对于表的要求

和ASP或者缓存组的要求是一样的，为：
* A primary key 或
* A unique index over non-nullable columns
另外，带压缩列的表不能复制。
VARCHAR2, NVARCHAR2, VARBINARY and TT_VARCHAR必须小于4M
BLOB 必须小于16M

多master复制的限制

多master的复制最典型的就是双向复制，当然不限于只有两个master。

检查表的复制冲突

在双向复制中，冲突检测可以检测的：
Such conflicts can be detected and resolved on a table-by-table basis by including timestamps in the replicated tables and configuring the replication scheme with the optional CHECK CONFLICTS clause in each table’s element description.

设置DATASTORE element的transmit durability

A master database configured for asynchronous or return receipt replication is durable by default. This means that log records are committed to disk when transactions are committed. The master database can be set to nondurable by including the TRANSMIT NONDURABLE clause in the element description.

Transaction records in the master database log buffer are, by default, flushed to disk before they are forwarded to subscribers. If the entire master database is replicated (ELEMENT is of type DATASTORE), you can improve replication performance by eliminating the master’s flush-log-to-disk operation from the replication cycle. This is done by including a TRANSMIT NONDURABLE clause in the element description. The TRANSMIT setting has no effect on the subscriber. The transaction records on the subscriber database are always flushed to disk.

Master databases configured for return twosafe replication are nondurable by default and cannot be made durable. Setting TRANSMIT DURABLE on a database that is configured for return twosafe replication has no effect on return twosafe transactions.

上面这段话比较重要，意思是说，如果是复制，不管数据库的设置是Durable还是Non-Durable，对于异步和return receipt复制，在复制前都会在本地写盘。但不是每一个commit都会写盘，而是累积到一定数量或每100ms写一次盘

见： A description of how the Log Buffer Flusher works (Doc ID 392247.1)

The log can also be flushed to disk by a variety of operations beyond an application issuing a Commit, such as a Durable Commit of another transaction, switching to a new log file or checkpointing the database. Activity by the replication agent can also cause log flushing. With the default TRANSMIT DURABLE replication option, when 256K of transactions is packaged up to be replicated, a local durable commit is also performed. This can mean replication causes the log buffer to be flushed more frequently. Replication also does a durable commit every 100 ms. The reason for this durable commit is to minimize the window for data loss and the overhead for master catchup. The log buffer can also be flushed by a read-only transaction, if it calls ttDurableCommit() (note: a read-only transaction that sets DurableCommit=1 in its connect string will not have this effect).

而return twosafe这时transmit non-durable的。因此，return two-safe的性能其实是优于return receipt的。

在classic复制中使用return service

就三种，NO RETURN, RETURN RECEIPT, RETURN TWOSAFE

classic复制的例子

首先声明，classic复制和ASP复制是不能并存的，ASP的standby是通过ttrepadmin克隆而成。而classic复制是两个预先建立好的独立数据库，而且表也需要预先建立

单向复制

cachedb1> create table a1(a int, primary key(a));
cachedb1> 
    CREATE REPLICATION repscheme
    ELEMENT e TABLE a1
    MASTER cachedb1 ON "timesten-hol"
    SUBSCRIBER cachedb2 ON "timesten-hol"
    RETURN RECEIPT;

在cachedb2上执行以上两个操作
在cachedb1和cachedb2上同时启动复制代理（说明一点，如果在启动复制前，两边的数据不一致，如记录数不等，是没有影响的）
cachedb1> call ttrepstart;
然后：
cachedb1> insert into a1 values(1);
cachedb2> select * from a1;
< 1 >
cachedb2> repschemes;

Replication Scheme TTHR.REPSCHEME:

  Element: E                              
  Type: Table TTHR.A1
  Master Store: CACHEDB1 on TIMESTEN-HOL Transmit Durable
  Subscriber Store: CACHEDB2 on TIMESTEN-HOL Return Receipt

  Store: CACHEDB1 on TIMESTEN-HOL
    Port: (auto)
    Log Fail Threshold: (none)
    Retry Timeout: 120 seconds
    Compress Traffic: Disabled

  Store: CACHEDB2 on TIMESTEN-HOL
    Port: (auto)
    Log Fail Threshold: (none)
    Retry Timeout: 120 seconds
    Compress Traffic: Disabled
    Return Service Wait Time: 10 seconds
    Return Service on Replication Stop: Disabled
    Return Service Failure Policy: (none)

一对多复制

CREATE REPLICATION twosubscribers
ELEMENT e TABLE ttuser.tab
    MASTER masterds ON "server1"
    SUBSCRIBER subscriber1ds ON "server2",
               subscriber2ds ON "server3";
如果使用不同的return service，可以用下例：
CREATE REPLICATION twosubscribers
ELEMENT e TABLE ttuser.tab
    MASTER masterds ON "server1"
    SUBSCRIBER subscriberds1 ON "server2" RETURN RECEIPT BY REQUEST
    SUBSCRIBER subscriber2ds ON "server3" RETURN RECEIPT
STORE masterds FAILTHRESHOLD 10;

不同的表复制到不同的目标

如a表由masterds到backup1ds，b表由masterds到backup2ds

CREATE REPLICATION twobackups
ELEMENT a TABLE ttuser.tab1
  MASTER centralds ON "finance"
  SUBSCRIBER backup1ds ON "backupsystem"
ELEMENT b TABLE ttuser.tab2
  MASTER centralds ON "finance"
  SUBSCRIBER backup1ds ON "backupsystem"
ELEMENT d TABLE ttuser.tab3
  MASTER centralds ON "finance"
  SUBSCRIBER backup2ds ON "backupsystem"
ELEMENT d TABLE ttuser.tab4
  MASTER centralds ON "finance"
  SUBSCRIBER backup2ds ON "backupsystem";

级联(Propagation )复制

CREATE REPLICATION propagator
ELEMENT a TABLE ttuser.tab
  MASTER centralds ON "finance"
  SUBSCRIBER propds ON "nethandler"
ELEMENT b TABLE ttuser.tab
  PROPAGATOR propds ON "nethandler"
  SUBSCRIBER backup1ds ON "backupsystem1",
             backup2ds ON "backupsystem2";

双向复制

在cachedb1和cachedb2上同时执行以下操作：
create table a2(a int, primary key(a));

CREATE REPLICATION r1
ELEMENT elem1 TABLE a2
  MASTER cachedb1 ON "timesten-hol"
  SUBSCRIBER cachedb2 ON "timesten-hol" RETURN RECEIPT
ELEMENT elem2 TABLE a2
  MASTER cachedb2 ON "timesten-hol"
  SUBSCRIBER cachedb1 ON "timesten-hol" RETURN RECEIPT;

在cachedb1和cachedb2上同时启动rep agent

cachedb1> select * from a2;
cachedb2> select * from a2;
cachedb1> insert into a2 values(1);
cachedb2> select * from a2;
< 1 >
cachedb2> insert into a2 values(2);
cachedb1> select * from a2;
< 1 >
< 2 >

双向复制时，应用有两种模式，一种是split workload，即两边是访问不同的数据。另一种是distributed workload，即双活，两边访问相同的数据，但必须应用保证冲突检测和解决。

混合复制

例如有些表为双向复制，有些表为单向复制。则必须定义在一个replication scheme中。不能定义两个schema，否则报错：
8169: Store CACHEDB1 ON TIMESTEN-HOL already transmits to store CACHEDB2 ON TIMESTEN-HOL; at most one path allowed from one store to another

以下为例子：

CREATE REPLICATION r1
ELEMENT elem1 TABLE a2
  MASTER cachedb1 ON "timesten-hol"
  SUBSCRIBER cachedb2 ON "timesten-hol" RETURN RECEIPT
ELEMENT elem2 TABLE a2
  MASTER cachedb2 ON "timesten-hol"
  SUBSCRIBER cachedb1 ON "timesten-hol" RETURN RECEIPT
ELEMENT e TABLE a1
    MASTER cachedb1 ON "timesten-hol"
    SUBSCRIBER cachedb2 ON "timesten-hol"
    RETURN RECEIPT;
cachedb1> repschemes;

Replication Scheme TTHR.R1:

  Element: E                              
  Type: Table TTHR.A1
  Master Store: CACHEDB1 on TIMESTEN-HOL Transmit Durable
  Subscriber Store: CACHEDB2 on TIMESTEN-HOL Return Receipt

  Element: ELEM1                          
  Type: Table TTHR.A2
  Master Store: CACHEDB1 on TIMESTEN-HOL Transmit Durable
  Subscriber Store: CACHEDB2 on TIMESTEN-HOL Return Receipt

  Element: ELEM2                          
  Type: Table TTHR.A2
  Master Store: CACHEDB2 on TIMESTEN-HOL Transmit Durable
  Subscriber Store: CACHEDB1 on TIMESTEN-HOL Return Receipt

  Store: CACHEDB1 on TIMESTEN-HOL
    Port: (auto)
    Log Fail Threshold: (none)
    Retry Timeout: 120 seconds
    Compress Traffic: Disabled
    Return Service Wait Time: 10 seconds
    Return Service on Replication Stop: Disabled
    Return Service Failure Policy: (none)

  Store: CACHEDB2 on TIMESTEN-HOL
    Port: (auto)
    Log Fail Threshold: (none)
    Retry Timeout: 120 seconds
    Compress Traffic: Disabled
    Return Service Wait Time: 10 seconds
    Return Service on Replication Stop: Disabled
    Return Service Failure Policy: (none)

总结

TimesTen支持两种复制策略，一个是ASP，一个是classic复制。
虽然ASP是推荐的，但classic复制支持双活，这是ASP做不到的，而且classic复制的粒度更细，更灵活，可以定义更复杂的拓扑，而ASP的复制拓扑是固定的。

参考

• A description of how the Log Buffer Flusher works (Doc ID 392247.1)