Storage Architecture

Disk-oriented

nessDB is a disk-oriented DBMS. All data are stored in nonvalatile storage after shutdown.

Foreign Keys

Not Supported

nessDB is a key-value DBMS so it does not support foreign key.

Storage Model

N-ary Storage Model (Row/Record)

nessDB v2.0 used a Small-Splittable Tree (SST), a variant of B-Epsilon-tree, for storage engine. SST has multiple levels with exponential expanding size. For level0, the data is un-sorted and the other levels are sorted. This data structure has good performance for random write.

Concurrency Control

Two-Phase Locking (Deadlock Detection)

nessDB v3.0 will use two-phase locking with deadlock detection, but this part does not finish yet. For deadlock detection, nessDB v3.0 will use Helgrind, a thread error detector that can detect potential deadlocks arising from lock ordering problems.

For nessDB v2.0, it is a single-thread engine. The engine can process a single request at a time.

Query Execution

Tuple-at-a-Time Model

nessDB is a key-value DBMS and each query only involves a single key-value pair.

Isolation Levels

Serializable

nessDB v3.0 support four isolation levels: read uncommitted, read committed, repeatable read and serializable.

Query Interface

Custom API

nessDB is an embedded DBMS with custom C API, as well as wrappers for Java, Python and Lua.

Compression

Naïve (Page-Level)

nessDB v3.0 used Snappy, a data compression and decompression library written in C++ by Google, to compress the buffered-tree node data when serializing the node to disk and decompress it when deserializing. nessDB v2.0 used QuickLZ, a compression library written in C, to compress the key-value pair.

System Architecture

Embedded

Data Model

Key/Value

nessDB is a key-value DBMS, which supports Redis-like protocol, such as SET, GET, DEL, EXISTS. To be specific, both key and value are general raw data and are represented by a pointer with length.

Stored Procedures

Not Supported

Views

Not Supported

Joins

Not Supported

nessDB does not support join. It is a key-value DBMS with only interfaces such as SET, GET, DEL, EXISTS.

Storage Organization

Heaps

For a tree node file, there is a header which stored the key-value pair in each level. And the header is followed by several var-length key-value pairs with data-offset.

Indexes

Log-Structured Merge Tree

nessDB used LSM-tree (Log-Structured Merge Tree) in v1.8, Small-Splittable Tree in v2.0, and will use fractal-tree in v3.0 for key-value storage. Those data structures are suitable for high-performance insertion compared to B-tree.

Checkpoints

Blocking

It does not fully support no redo log and recovery. nessDB v3.0 will support sharp checkpoint with a 60 seconds period according to the code, but the recovery part has not been finished.