History[05]

The development of the WiredTiger storage engine started in November 2018. The corresponding company WiredTiger Software was founded in 2012 and was later acquired by MongoDB to replace its original mmap-based storage engine as the default storage engine.

Checkpoints[06][07]

Checkpoints in WiredTiger are different from typical checkpoints in other database systems. In WiredTiger, the block storage manager is actually managing multiple versions of checkpoints, i.e. all the durable data in WiredTiger are considered as checkpoints. Checkpoint process in WiredTiger are triggered every 60 seconds or manually triggered.

A successful checkpoint in WiredTiger requires four phases. We will take B-tree as an example in the following discussion of these phases. First, all dirty leaf pages are written to disk. Then all dirty internal pages are written to disk. Then fsync is called on the checkpoint file so that the checkpoint is persistent in disk. Finally, the new root of the B-tree is atomically changed to the new root. This is somewhat similar to shadow paging. When a checkpoint is deleted, WiredTiger will check whether there is any dependency between the two adjacent checkpoints and merge them if there are common pages in the two B-trees.

Compression[08][09]

Multiple compression techniques are used in WiredTiger.

Block-level: Existing compression libraries can be directly plugged into WiredTiger. It supports LZ4, snappy, zlib, and Zstd.

Record-level: WiredTiger supports dictionary encoding, run-length encoding, prefix encoding, and Huffman encoding.

Concurrency Control[10]

WiredTiger employs multi-version concurrency control (MVCC), in which readers never block writers. In case of write-write conflict, one transaction will be rolled back and retried. Three isolation levels are provided: READ_UNCOMMITTED, READ_COMMITTED, and SNAPSHOT. READ_COMMIT is the default isolation level. In addition, transactions' working set is required to fit in main memory in WiredTiger.

In WiredTiger, transactions can pin particular versions of the database, which are called "named snapshots". Queries within such transactions are executed as if they are operating on the named snapshots.

Indexes

Storage Architecture

System Architecture