QuestDB

View Current Viewing Revision #9 from 09/14/2020 9:39 a.m.

QuestDB is an open-source SQL database designed to process time-series data, faster.

It uses a column-oriented approach, vectorized execution, SIMD instructions, and a whole array of low-latency techniques. The whole code base is built from scratch and without dependencies, in the name of performance. QuestDB is 100% free from garbage collection.

QuestDB implements SQL, and augments it for time-series. It exposes a Postgres wire protocol, a high-performance REST API, and even supports ingestion with InfluxDB line protocol. It supports both relational and time-series joins, which makes it easy to correlate data over time. Writes are durably committed to disk, meaning that the data is safe - yet instantly accessible.

History

QuestDB was born as a side project from Vlad Ilyushchenko who applied principles from low latency techniques found in the financial services industry. QuestDB is backed by YCombinator and its recent Hacker News launch goes through the story of QuestDB since 2013.

System Architecture

Shared-Nothing Embedded

Concurrency Control

Timestamp Ordering

QuestDB uses a single writer model. The writer re-orders timestamps on the fly. Multi-publisher, single consumer queue provides concurrent write access.

Query Execution

Vectorized Model

SIMD-based vector execution of queries.

Indexes

Hash Table

Indexing is available for symbol columns. Index support for other types will be added over time.

Stored Procedures

Not Supported

Storage Model

Decomposition Storage Model (Columnar)

QuestDB uses a column-based storage model. Data is stored in tables with each column stored in its own file and its own native format. New data is appended to the bottom of each column to allow data to be organically retrieved in the same order that it was ingested.

Isolation Levels

Read Committed Repeatable Read Read Stability

Joins

Hash Join Semi Join

QuestDB supports the following types of joins: INNER, OUTER, CROSS, ASOF and SPLICE. FULL joins are not yet implemented and are on our roadmap. All supported join types can be combined in a single SQL statement; it is job of QuestDB SQL optimiser to determine best execution order and algorithms.