History[04][05]

QuestDB started as a side project in 2014 by Vlad Ilyushchenko who applied principles from low latency techniques found in the financial services industry. QuestDB is backed by YCombinator.

Concurrency Control[06]

QuestDB uses a single writer model. The writer re-orders timestamps on the fly. Multi-publisher, single consumer queue provides concurrent write access.

Data Model[07]

QuestDB uses a relational model with column-based storage model.

Foreign Keys

Hardware Acceleration

SIMD optimised analytics.

Indexes[08]

Indexing is available for symbol columns. The symbol datatype in QuestDB is used to store repeatitive strings. Internally, it is stored as a dictionary of integers and corresponding string values. The Index support for other types will be added over time.

Isolation Levels[09]

QuestDB uses the READ COMMITTED isolation level.

Joins[10]

QuestDB supports INNER, LEFT OUTER, and CROSS join types. FULL joins are not yet implemented and are on our roadmap. In addition, QuestDB also supports ASOF, LT, and SPLICE join types particularly useful for time-series analytics.

Joins in QuestDB are internally implemented as nested loop joins and hash joins.

Logging[11]

Starting from version 7.0, QuestDB supported using WAL to ingest data. With the addition of WAL-supported tables, clients can write to disk independently without holding the lock of the table writer. Transaction numbers are provided by a central sequencer protecting a single source of truth. After applying the WALs, the data is applied to the table asynchronously by another job.

Parallel Execution

Multi threaded SQL execution

Query Compilation[12][13]

QuestDB uses a JIT compiler to improve performance by compiling the predicate evaluation part of the query. After determining whether a filter is suitable for compilation, the compiler frontend would transform the abstract syntax tree (AST) of the filter into an intermediate representation (IR). The IR is then processed by the compiler backend to emit vectorized machine code with the AVX2 instruction set.

Query Execution[14]

QuestDB uses the vectorized execution model and takes advantage of SIMD operations for fast aggregations and predicate evaluation.

Query Interface[15][16]

QuestDB has a SQL interface mostly compatible with the PostgreSQL dialect. It also extends the standard SQL with constructs simplifying the semantics of time-series queries (e.g. LATEST ON, SAMPLE BY). It supports a simple REST API, the Postgres wire protocol, as well as the InfluxDB line protocol (ILP). QuestDB now has an ILP client library in the following languages: .Net, C, C++, Go, Java, JavaScript, Python, and Rust. It is compatible with other third-party tools including Flink, Grafana, Kafka, MindsDB, Pandas, Prometheus, Redpanda, and Telegraf.

Storage Architecture[17]

QuestDB uses a disk-oriented storage architecture with memory-mapped file data access.

Storage Model[07]

QuestDB uses a column-based storage model. Data is stored in tables with each column stored in its own file and its own native format. New data is appended to the bottom of each column to allow data to be organically retrieved in the same order that it was ingested.

Storage Organization

Stored Procedures

System Architecture

Views