TDSQL

TDSQL is an abbreviation for TecentDistributed SQL. It is derived from MySQL and aimed at providing digital payment services for government and financial companies. TDSQL supports a strong consistency model based on Raft. Data are partitioned horizontally based on the partition key and auto sharding is enabled to facilitate data migration. TDSQL can be deployed on both private cloud and public cloud.

History

TDSQL was initially used internally (2012) by the Tecent company. It was used as the DBMS for WeBank (WeBank is China’s first digital bank established in December 2014) in 2014. TDSQL is moved to the cloud and starts to provide services for companies other than Tecent in 2015.

Query Interface

HTTP / REST

TDSQL supports SDK for python, java, php, nodejs, .net. Tecent Cloud CLI and HTTPS/REST API can also be used to interact with the DBMS

Checkpoints

Blocking

Memory data need to be flushed to disk to make sure all data are saved. Read locks are need to lock on all tables before making the checkpoint. The checkpoint is stored in the cloud storage, you need to create the cloud storage before flushing all data to the disc.

Data Model

Relational Key/Value Document / XML

TDSQL supports multiple data models to meet different application requirements. MySQL API, Redis API, and MongoDB API are used with SQL execution engine, KV execution engine and Doc execution engine respectively. There is a data conversion layer between the execution layer and the storage layer to unify the data format.

Indexes

B+Tree

TDSQL uses the index supported by InnoDB. InnoDB uses B+ tree as the default index data structure.

System Architecture

Shared-Nothing

The TDSQL framework can be divided into the following layers. Computer layer, which consists of the compute engine, is stateless. For the storage layer, data are stored in the unit of the replica set with multiple duplicates. Strong consistency is guaranteed by the Raft consistency model.

Joins

Not Supported

It only supports joining on the same node, which requires join tables to be partitioned on the same key or the table is small enough to be copied across all partitions. Then the proxy/coordinator merges the results from different shards to generate the final result. Join is not supported if the join operation touches data across nodes and the join key is different from the partition key.

Concurrency Control

Two-Phase Locking (Deadlock Detection)

Global wait-for-graphs are created to detect deadlocks. The timer is applied to count how long a transaction has been started. If a transaction is not finished within a given time period, the timer will indicate a potential deadlock.

Storage Model

N-ary Storage Model (Row/Record)

Since TDSQL is derived from InnoDB, each node uses the same storage model as InnoDB, which is a row-storage DBMS.

Storage Architecture

Disk-oriented

3-level storage strategy is used to save storage resources and reduce query latency. According to the 3-level storage strategy, all data can be classified into Standard Storage (hot data), Infrequent Access Storage (warm data), and Archive (cold data). Automatic conversion is trigger between different storage levels under different lifecycles. Standard Storage is stored in three copies on SSD and Infrequent access Storage is stored in 2 copies on SATA. Archive is stored in 1.3 copies on SATA.

TDSQL Logo
Website

http://tdsql.org/

Country of Origin

CN

Start Year

2012

Project Type

Commercial

Derived From

MySQL

Inspired By

MySQL

Compatible With

MongoDB, MySQL, RocksDB