Checkpoints

Not Supported

Dolt does not support mutable database files, hence it does not explicitly take checkpoints. Dolt has a manifest that stores pointers to all currently active table files, which is updated atomically on every mutation of the database.

Compression

Naïve (Page-Level)

Dolt uses Snappy an open-sourced compression library that prioritizes speed over size. All chunks are compressed with Snappy before storage and are decompressed as they are read into the block cache. It is necessary to decompress data in order to process queries.

Concurrency Control

Multi-version Concurrency Control (MVCC)

Dolt supports transactions. Transactions use git-style merge semantics for commit.

Data Model

Relational

Dolt emulates MySQL.

Foreign Keys

Supported

Dolt emulates MySQL's foreign key support.

Indexes

B+Tree

Dolt supports a B-tree like index structure for table primary keys. Configurable single or multi column secondary indexes of the same structure are also supported.

Isolation Levels

Repeatable Read

Dolt supports the REPEATABLE_READ isolation level for its transactions. Additionally, clients can connect to different branches of the same database, with each branch being completely isolated from all others. Other isolation levels are in development.

Logging

Not Supported

Dolt does not support logging but ensures data durability. New data is written once to new table files containing new chunks. The table files are written to disk before the manifest references are updated.

Query Compilation

Not Supported

Query Execution

Tuple-at-a-Time Model

Dolt supports iterator based query processing with no intra-query parallelism.

Query Interface

Custom API SQL Command-line / Shell

Dolt provides a Git like command-line interface. Dolt also allows SQL queries through the MySQL wire protocol and through its command line. Dolt also allows data to be imported and exported using CSV files.

Storage Architecture

Disk-oriented

Dolt uses disk storage and can currently store datasets up to 100GB.

Storage Model

N-ary Storage Model (Row/Record)

Dolt stores tables in the N-ary Storage Model with clustered primary keys. The entire dataset is content-addressed as a Merkle Tree of component blocks. A Merkle tree is a hash-based data structure that is a generalization of the hash list. It is a tree structure in which each leaf node is a hash of a block of data, and each non-leaf node is a hash of its children. The boundaries for internal and leaf nodes are chosen by a rolling hash of the block contents.

Storage Organization

Sorted Files

Dolt stores its dataset as a Merkle Tree of component blocks. The content-addressed blocks are stored in write-once table files with a static binary-searchable index at the end. When the table files grow to a number beyond a certain threshold, a compaction phase is run. New data is only written once. It is written to new table files containing the new chunks. These table files are flushed to disk before the manifest referencing them is updated.

Stored Procedures

Supported

Dolt mimics MySQL stored procedures.

System Architecture

Shared-Everything Embedded

Dolt is not distributed at a system level. Dolt is designed to distribute the same database to multiple locations where it can be worked on in isolation and any edits from one location can be explicitly pulled or pushed to another location.

Views

Virtual Views

Dolt supports Views. Views are versioned as part of the schema.