Storage Model

N-ary Storage Model (Row/Record)

Dqlite stores tuples in files backed up by disk as supported by the underlying SQLite engine. In addition, the tuples format consisted of a header and a body. The header of size 64-bit is a collection of 4-bit code which specifies the type of the corresponding value of the tuple. The body consists of values of the tuple that follow immediately after the header.

Data Model

Relational

Since the underlying database engine is SQLite, Dqlite only supports row-based relational data model.

Storage Architecture

Disk-oriented

Concurrency Control

Deterministic Concurrency Control

Dqlite uses Raft protocol to keep all the replicas in sync. Since only the leader node in Raft will be able to add WAL entry, while other followers will replicate the WAL entries from the leader, there will not be conflicting WALs proposed by multiple nodes. In cases when a client tries to perform a write transaction on a non-leader node, the transation will fail.

In addition, each dqlite node runs in a single thread that runs in a loop to execute query on the underlying SqLite engine.

When not enough nodes are available in light of network parition, writes to the database might hung until consensus is reached or a timeout if triggered so that the write fails.

Checkpoints

Blocking

Similar to SQLite, Dqlite performs checkpointing automatically when the length of the Write-Ahead-Logging (WAL) file reaches a threshhold size. (The default value in the Dqlite setting is 1000 pages). The leader will delay the checkpoint operation if the WAL logs have not reached the threshhold or the underlying SQLite database is "locked" by another connection and not available for flushing Upon performing the checkpoint, the leader will issue a checkpoint command and sends that command to the followers. Once the checkpointing command is committed by all the followers, they will perform checkpointing respectively in blocking mode.

System Architecture

Shared-Nothing

Query Execution

Tuple-at-a-Time Model

Upon executing a SQL statement, the request will be dispatched to the leader node's execution loop first, which will invoke the underlying SQLite engine to step over the statement. If the execution of statement requires actions across replicas, the control will be switched back to the main loop. After the main loop finishes replicating the Raft logs across replicas, the control will be switched back to the execution loop, and thus the underlying SQLite engine will continue stepping over.

Query Interface

SQL

Storage Organization

Heaps