QuestDB

View Current Viewing Revision #23 from 04/02/2023 midnight

QuestDB is an open-source time-series relational database. It uses a column-oriented approach and supports heavy parallelized vectorized execution using SIMD instructions. QuestDB implements a SQL interface and augments it with extensions simplifying the semantics of time-series queries. It supports high-throughput, schema-agnostic ingestion using the Postgres wire protocol and the InfluxDB line protocol. QuestDB is written from scratch in Java and C++ with no external dependencies and zero garbage collection.

History

QuestDB started as a side project in 2014 by Vlad Ilyushchenko who applied principles from low latency techniques found in the financial services industry. QuestDB is backed by YCombinator.

Concurrency Control

Timestamp Ordering

QuestDB uses a single writer model. The writer re-orders timestamps on the fly. Multi-publisher, single consumer queue provides concurrent write access.

Data Model

Relational

QuestDB uses a relational model with column-based storage model.

Foreign Keys

Not Supported

Hardware Acceleration

Custom

SIMD optimised analytics.

Indexes

Hash Table

Indexing is available for symbol columns. The symbol datatype in QuestDB is used to store repeatitive strings. Internally, it is stored as a dictionary of integers and corresponding string values. The Index support for other types will be added over time.

Isolation Levels

Read Committed Repeatable Read Read Stability

Joins

Nested Loop Join Hash Join Semi Join

QuestDB supports INNER, LEFT OUTER, and CROSS join types. FULL joins are not yet implemented and are on our roadmap. In addition, QuestDB also supports ASOF, LT, and SPLICE join types particularly useful for time-series analytics.

Logging

Physical Logging

Starting from version 7.0, QuestDB supported using WAL to ingest data. With the addition of WAL-supported tables, clients can write to disk independently without holding the lock of the table writer. Transaction numbers are provided by a central sequencer protecting a single source of truth. After applying the WALs, the data is applied to the table asynchronously by another job.

Parallel Execution

Inter-Operator (Vertical)

Multi threaded SQL execution.

Query Compilation

JIT Compilation

Query Execution

Vectorized Model

SIMD-based vector execution of queries.

Query Interface

SQL

QuestDB enhances ANSI SQL with time series extensions to manipulate time stamped data.

Storage Architecture

Disk-oriented

Memory mapped files data access.

Storage Model

Decomposition Storage Model (Columnar)

QuestDB uses a column-based storage model. Data is stored in tables with each column stored in its own file and its own native format. New data is appended to the bottom of each column to allow data to be organically retrieved in the same order that it was ingested.