Compression

Dictionary Encoding

OlegDB uses the LZ4 lossless data compression algorithm for storing values on disk. The user has the possibility to disable this feature if they want to accelerate data operations.

Concurrency Control

Not Supported

The `concurrent label' of OlegDB comes from its frontend layer, which is able to accept multiple connections at the same time. OlegDB does not technically support concurrent transactions at the backend layer.

Data Model

Key/Value

At its core, OlegDB is simply a concurrent key/value hash table. It uses the open-source MurmurHash3 hash function (32 bits option).

Joins

Not Supported

OlegDB does not support joins.

Logging

Logical Logging

OlegDB uses an append-only log file (nicknamed AOL) to record the high level operations executed by transactions on the database. The log records are designed to be human readable.

Query Interface

HTTP / REST

Communication with OlegDB is done via a simple HTTP API. The following HTTP method are supported:

• POST: create new records/update records
• GET: fetch records
• DELETE: perform deletions
• HEAD: retrieve additional information about the records (e.g expiration date of a key)

Storage Architecture

Disk-oriented

OlegDB uses disk storage. The system keeps track of its state through two different types of data:

• the Value file: contains all the inserted values, aligned in a four megabytes block (by default). On startup, this file is loaded in RAM using the mmap() system call.
• the AOL file: contains transaction logs. It is regularly cleaned to remove expired data.

In the original release of OlegDB, this AOL file was the only persistent data of the system and all the values were kept in memory while the database was running. The Value file was added in later versions of the implementation.

OlegDB handles collisions via separate chaining (linked list). At runtime, a separate bucket chains list is stored in volatile memory, where each chain (linked list) corresponds to a specific key. Each bucket of the chain contains information about a unique value mapping to the chain's key (size, location in the Value file ... etc.). When the system reaches the limit of the allocated memory for this structure, a new block is allocated, and all records in the database are rehashed. The default size amount allocated by OlegDB is a tweakable parameter.

System Architecture

Shared-Everything

OlegDB is not a distributed database. It is built to run on a single node. The HTTP frontend layer can however serve several OlegDB database instances running at the same time on the same machine.