Checkpoints

Not Supported

ReductStore currently does not support checkpoints.

Compression

Naïve (Record-Level)

ReductStore directly writes blobs to disk using MessageLite::SerializeToOstream(std::ostream * output) function from Protocol Buffers (Google), which by default does not do any compression.

Concurrency Control

Not Supported

Queries can either be submitted from ReductStore's C++/Python/JavaScript client SDK, or an HTTP API. Query strings are parsed and passed in as protobuf messages. Executor tasks are created from query strings strictly in FIFO order of their arrival time. There is no guarantee that tasks will run in the same order as they arrive.

Data Model

Key/Value

ReductStore maintains three Data abstractions: Entry (logical), Block (physical), and Bucket (logical). Each Entry is a blob associated with a key and a timestamp. Each Block physically contains one or multiple Entries of the same timestamp and is backed by a single file on the file system. Entries that are within the same bucket and smaller than the file system's minimum file size are grouped and stored together into a single Block (backed by one file) to reduce the storage overhead of small files. A Bucket logically contains one or multiple physical blocks and is typically used to separate time series data by topic or user.

Buckets enforce storage limits (the total blob size and the total number of entries within the Bucket) and write behavior (reject write or delete earliest) when the storage limit is reached.

Deleting a single Entry or a Block is not supported. The minimum deletion level is a Bucket.

Foreign Keys

Not Supported

ReductStore does not have the concept of a table.

Indexes

Not Supported Block Range Index (BRIN)

Index is not supported. A BRIN-like storage hack is implemented.

Block begin time is defined as the creation time of the block, with microsecond precision. Multiple logical blocks with the same begin time are grouped and stored as a single file on disk. Block start timestamp is embedded in the filename for range queries to perform binary search to locate the first relevant block.

Isolation Levels

Not Supported

ReductStore is not a transactional database. No isolation control mechanism is implemented.

Joins

Not Supported

Only range queries and continuous queries are supported.

Logging

Not Supported

No logging mechanism is implemented. Pending writes are all lost on system crash.

Query Compilation

Not Supported

Query Execution

Tuple-at-a-Time Model

Each executor scans blocks from files one entry at a time. Executors are allowed to read/write blocks that reside in different files concurrently.

Query Interface

Custom API HTTP / REST

Provides HTTP methods for Entry level read, write, and range query. Provides HTTP methods for Bucket level metadata read, metadata update, create, and remove. Provides HTTP methods for Server level statistics read, bucket statistics read, and engine health check. Provides HTTP methods for token-based access control.

Storage Architecture

Disk-oriented

Storage Model

N-ary Storage Model (Row/Record)

Storage Organization

Sorted Files

Blocks are sorted by timestamp, grouped by microsecond level begin time, and stored as individual files on disk. Binary search is used to locate starting entry for range queries.

Stored Procedures

Not Supported

System Architecture

Shared-Everything

ReductStore is intended for single node deployment.

Views

Not Supported