Database management-phase 250 deep introduction of multi-master and Multi-Active database technology-Cantian storage engine (I)(20241009)

author: fat head Fish Tank (Yin Haiwen)
Oracle ACE Pro: Database(Oracle and MySQL)
PostgreSQL ACE Partner
10 years of experience in the database industry, now mainly engaged in database services
with OCM 11g/12c/19c, MySQL 8.0 OCP, Exadata, CDP and other certifications
MVP of motianlun, star of mojili of the year, ITPUB certified expert, member of expert group of 100, OCM lecturer, technical consultant of PolarDB open source community, external technical consultant of HaloDB, member of OceanBase observer mission, technical Adviser, MOP technical community of youth society (youth database learning Mutual Aid Association)
the circle has the titles of "director", "security guard", "the biggest enemy of domestic database" and so on. It is not a famous social terrorist (social terrorist)
public number: fish tank of fat head fish; CSDN: fish tank of fat head fish (Yin Haiwen); Mo Tianlun: fish tank of fat head fish; ITPUB:yhw1809.
Except for authorized reprinting and indicating the source, all are "illegal" plagiarism.

At the invitation of towering, I gave an online speech and discussion on the architecture of database, RDMA and memory separation in Huawei through Huawei Jixian community. After a period of time, it is still necessary to understand Cantian.

1 Introduction

Cantian is a storage engine that uses a separate architecture of storage and calculation. It uses key technologies such as distributed cache technology, transaction MVCC mechanism, and high availability of multi-master clusters, it enables a common standalone database to have multiple read and write capabilities similar to Oracle RAC. The Cantian engine can be loaded and run by databases such as MySQL in a non-intrusive manner without modifying the implementation of existing databases. The multi-read and multi-write engine of Cantian needs to be built based on shared storage. Any storage system that can provide shared storage and standard file interfaces can be connected.
Main features of Cantian:

• read more and write more

• cluster management: cluster member status maintenance, cluster exception handling, and arbitration.

• Supports connection with MySQL: Compatible with Innodb

• supports MySQL shared system tables

• supports rolling upgrades.

• Supports backup

Cantian was officially open-source at the Huawei data storage user elite Forum on August 25, 2023, using the Mulan open-source license agreement. Code Repository: https://gitee.com/openeuler/cantian
currently, the Klustron database has released the Cantian partner version.

2 engine composition

the Cantian engine consists of five main parts:

• CTC(Cantian Connector): Supports the Cantian engine as a storage engine plug-in for distributed databases. It supports database functions such as DDL, DML, and transaction, and is compatible with the ecosystem application of distributed databases.

• CMS(Cluster Manager Service): manages clusters.

• Multi-read and multi-write module: the Cantian engine is a multi-write cluster based on shared storage. Each node is structurally equivalent, the multi-read and multi-write module ensures that DDL, DML, and DCL operations can be performed on the database from any node. Changes made by any node can be seen by other nodes when the isolation level is met. All compute nodes share and read and write the same user data on the storage.

• Storage Access layer: the Cantian engine performs read and write operations on data through shared storage interfaces, such as standard file interfaces, and is mutually exclusive by using the lock mechanism provided by the file system.

• Tools: including backup and recovery tools: export all tables in the database to SQL statements or table text, and import logical data files in text format to the database during logical recovery. O & M management tools: each command line.

3 engine architecture

in the architecture, yellow is the core module of Cantian, and the blue part is not included in the open source code of Cantian. It needs to be provided by Open Source users of Cantian, such as database vendors.

• (1) it is the Connector part connected to MySQL, which runs in the same process Space. Its main function is to accept MySQL requests and communicate with Cantian processes through IPC. (This part of the code fully complies with MySQL's open source protocol for open source)

• ② provide ctmgr to be called by a third-party management Agent and reported to the management platform

• ③ provide ctbackup tool, which is syntactically compatible with extrabackup. The backup software schedules the backup operation.

• ④ provide scripts such as installation and deployment, disaster recovery configuration, etc. The management platform calls to complete operations such as installation, scale-out, and disaster scheduling.

• ⑤ provide instructions on installing containers. Resource distribution management, container management platform.

• The Cantian engine consists of the Cantian business process and the cluster management CMS process. Each Cantian process corresponds to a CMS process to supervise the Cantian operation status, split-brain arbitration, and fault initiation. When the CMS fails, the watchdog is pulled up. High-speed network communication between Cantian running on multiple servers. Cantian accesses shared storage to obtain data.

The Global Cache mechanism of Cantian is similar to that of RAC of the earlier version. It determines the Master instance of a buffer block. When an instance has the corresponding buffer block to be accessed, it can be directly accessed, otherwise, you need to query the location of the buffer block from the Master instance, and the Master notifies the buffer block instance to send it to the database instance that needs to access the data.

For more information about architecture, see the technical white paper in the code warehouse for detailed introduction.

4 File Distribution

similar to Oracle RAC, the file location in Cantian is stored in shared storage,:

• each instance in the cluster has exclusive access to log, undo, temp, and other files. Only when the fault is recovered can other instances in the cluster read

• table spaces such as control files, system, and users can be read and written concurrently in the cluster.

5 distributed MVCC

1. segment management
   according to the file distribution in the previous section, each instance has its own undo tablespace, that is, it has its own independent undo segment management and recycling mechanism. Generally, this instance can only be read and modified, data fusion is not enabled, and instances do not interfere with each other.
   Cantian uses the SMON thread background to shrink within the undo segment of the current instance.

2. txn management
   (xmap, xnum) xmap -> (seg_id, slot)
   the transaction xid is separated with the seg_id. Based on the seg_id in the current transaction xid, you can quickly locate the instance where the transaction is located and the segment in the actual instance.
   Each instance has an independent transaction table under the undo. The transaction allocation is based on the single-machine allocation and recycling mechanism. Transactions cannot be reused between instances.

3. Distributed MVCC
   in a consistent read scenario in cluster mode, the page to be queried by an instance may be on another instance or even being modified by transactions on another instance. Because the undo and transaction before the instance are isolated, the current query cannot directly build the CR page locally. If you rely on other instances to transmit the required undo page to the local through DCS, the efficiency is too low. Consider using the method of transmitting CR pages, and the relevant instance background participates in the construction of CR pages.
   
   When MVCC requires a PRE-IMAGE that has been changed multiple times by multiple instances, the construction process in the Cantian engine is relatively complicated. You need to use xid to determine the instance information corresponding to undo, after multiple instances are passed forward, the corresponding CR page can be constructed. Based on the previous Global Cache mechanism, if the page is not local, build a CR page on the instance where the owner is located.

6 limits/requirements

for the content involved in this issue, Cantian has the following restrictions or requirements:

• currently, the released version only supports the 2-node deployment mode.

• A stable and high-speed communication network is required between compute nodes to exchange page, lock, and other resources.

Summary

as a storage engine, Cantian is similar to Oracle RAC's ASM, which enables single-point databases to use high-performance and highly reliable shared storage to achieve horizontal expansion of computing performance.
As usual, I know what I have written.