ReductStore

View Current Viewing Revision #4 from 03/31/2023 11:45 p.m.

ReductStore is a time-series database designed specifically for blob (binary) data.

Checkpoints

Compression

ReductStore packs blobs that are smaller than the minimum size of file system blocks into larger logical blocks that are stored as files to reduce wasted disk space. No compression algorithm is applied at any level.

Concurrency Control

Not Supported

Executor tasks are created from query strings strictly in FIFO order of their arrival time. There is no guarantee that executors will run in the same order as they're created.

Data Model

Key/Value

Each entry is a blob associated with a key at creation time. Entries smaller than the file system's minimum file size are grouped and stored together in a logical block, which is backed by a single file. A logical bucket contains multiple logical blocks. Bucket ensures storage limits (total blob size and the total number of records) and write behavior when the storage limit is reached (reject write or delete earliest).

Deleting a single entry is not supported. The minimum deletion level is a single bucket.

Foreign Keys

Not Supported

Indexes

Block Range Index (BRIN)

Block begin time is defined as the creation time of the block, with microsecond precision. Multiple logical blocks with the same begin time are grouped and stored as a single file on disk. Block start timestamp is embedded in the filename for range queries to perform binary search to locate the first relevant block.

Isolation Levels

Not Supported

No isolation control mechanism is implemented. Effectively Read Uncommitted.

Joins

Not Supported

Only range queries and continuous queries are supported.

Logging

Not Supported

No logging mechanism is implemented. Pending writes are all lost on system crash.

Query Compilation

Not Supported

Query Execution

Tuple-at-a-Time Model

Each executor scans blocks from files one entry at a time. Executors are allowed to read/write blocks that reside in different files concurrently.

Query Interface

Custom API HTTP / REST

Provides HTTP methods for Entry level read, write, and range query. Provides HTTP methods for Bucket level metadata read, metadata update, create, and remove. Provides HTTP methods for Server level statistics read, bucket statistics read, and engine health check. Provides HTTP methods for token-based access control.