History[04][05][06]

ReductStore is a personal project started by Alexey Timin in 2021 with no publicly stated derivations or inspirations from other existing database systems. It has since received monthly releases and bug fixes. Alexey Timin remains the sole major contributor to this project. The project is currently undergoing a major rewrite with Rust and is actively seeking new collaborators to join.

Checkpoints

ReductStore currently does not support checkpoints.

Compression[07][08][09]

ReductStore directly writes blobs to disk using MessageLite::SerializeToOstream(std::ostream * output) function from Protocol Buffers (Google), which by default does not do any compression.

Concurrency Control[10]

Queries can either be submitted from ReductStore's C++/Python/JavaScript client SDK, or an HTTP API. Query strings are parsed and passed in as protobuf messages. Executor tasks are created from query strings strictly in FIFO order of their arrival time. There is no guarantee that tasks will run in the same order as they arrive.

Data Model[11][12]

ReductStore maintains three data abstractions: Entry (logical), Block (logical, physical), and Bucket (logical). Each Entry is a blob associated with a key and a timestamp. Each logical Block contains one or multiple Entries of the same timestamp. Logical Blocks with the same microsecond level begin time are grouped and stored as a single physical Block backed by a single file on disk. Entries that are within the same bucket and smaller than the file system's minimum file size are grouped and stored together into a single Block (backed by one file) to reduce the storage overhead of small files. A Bucket logically contains one or multiple physical blocks and is typically used to separate time series data by topic or user.

Buckets enforce storage limits (the total blob size and the total number of entries within the Bucket) and write behavior (reject write or delete earliest) when the storage limit is reached.

Deleting a single Entry or a Block is not supported. The minimum deletion level is a Bucket.

Foreign Keys

ReductStore does not support foreign keys.

Indexes[13]

Index is not supported. A BRIN-like storage hack is implemented.

Multiple logical blocks with the same microsecond-level start timestamp (earliest timestamp of all its Entries) are grouped and stored as a single file on disk. Block start timestamp is embedded in the filename for range queries to perform binary search to locate the first relevant block.

Isolation Levels

ReductStore is not a transactional database. No isolation control mechanism is implemented.

Joins[14]

Only range queries and continuous queries are supported.

Logging

ReductStore does not support logging currently.

Parallel Execution

Not supported. Each query will only be assigned to a single executor.

Query Compilation

ReductStore does not support and does not benefit from query compilation currently. Only range queries and continuous queries are supported, which always performs binary search and/or linear search to locate the starting/ending files for read/write.

Query Execution[07]

ReductStore adopts Entry-at-a-Time model.

Query Interface[15]

Provides HTTP methods for Entry level read, write, and range query. Provides HTTP methods for Bucket level metadata read, metadata update, create, and remove. Provides HTTP methods for Server level statistics read, bucket statistics read, and engine health check. Provides HTTP methods for token-based access control.

Storage Architecture[13]

ReductStore is a shared-nothing disk-oriented time-series database that directly persists/retrieves Entries to/from disk and does not cache any intermediate results in memory during query execution.

Storage Model[16]

ReductStore adopts a hybrid storage model. Within a single physical Block, all metadata fields (timestamps, user-defined labels, file offsets) of all Entries are stored together in a row-oriented fashion as the file header, while blobs of all Entries are stored consecutively in the remaining part of the file.

Storage Organization[17]

Entries within each logical Block are sorted by timestamp. Logical Blocks are sorted by earliest entry timestamp, grouped by microsecond-level granularity into a single physical Block, and stored as a single file on disk.

Stored Procedures

System Architecture[03]

ReductStore is intended for single-node deployment.

Views