SwayDB is an embedded, non-blocking and type-safe NoSql database under development. It can be configured for both persistent and in-memory storage. It supports normal operations for key-value stores, and additionally supports auto expiring key values. It provides a Scala and Java API. It reaches up to 600,000 writes per second for in-memory databases and up to 300,000 for persistent databases. For transactions, SwayDB only guarantees atomicity. Log-structured merge-tree is the main underlying algorithm.
No checkpoints are supported. For its in-memory implementation, no checkpointing or logging is supported. For the persistent version, it can persist its memory segments to disk, but this is used for reducing that level's storage, not for recovery.
It has a notion of 'eventual persistent' in its documentation. It means that the database can write the memory storage to disk whenever it exceeds a threshold. This is somewhat similar to checkpointing, but it is not a full checkpoint, so the data left in memory can still be lost.
Naïve (Page-Level) Prefix Compression
Segments are chunks of key-value pairs in a Log-Structured Merge Tree. Key-values within the same segment can be grouped to create sub-segments and then be compressed by specifying groupingStrategy. SwayDB supports compressing them using LZ4 and Snappy, which are open-sourced Java compression tools. Strictly-speaking, the compression is done on sub-segment level instead of page level.
Additionally, duplicate data within index entries are always compressed. Duplicate values within same segments are also detected and compressed. All index entries are prefix-compressed with the previous index entry.
SwayDB assumes it runs 'under a single process in a single application', as the author clarifies on Reddit. Everything is atomic in SwayDB, including transactions. The only guarantee it has on transactions is atomicity. No concurrency control is needed, and indeed it is not mentioned in docs at all.
SwayDB supports both in-memory and disk storage. For its in-memory version, it does not provide guarantee on data durability because it does not have logging. However, the user can persist the memory to disk at some time.
For its persistent database, write-ahead logging is used for durability. Data is organized into segments on different levels, aligning to the Log-Structured Merge Tree algorithm.
Decomposition Storage Model (Columnar)
For the segment file format, SwayDB stores all values in the head of the file, in the same ordering as keys. Key index entries are stored after the value bytes. At the last of the file is the checksum, bloom-filter and other metadata about the segment.
Log-Structured Merge Trees are used in SwayDB. Storage are divided into different levels, corresponding to different hardware capabilities. If the upper level cannot find an entry, it will go into the next level. During a compaction, data from upper levels are pushed into lower levels, and deletion is also performed if necessary.
https://github.com/simerplaha/SwayDB
Simer Plaha
2018