Concurrency Control

Timestamp Ordering

QuestDB uses a single writer model. The writer re-orders timestamps on the fly. Multi-publisher, single consumer queue provides concurrent write access.

Data Model

Relational

QuestDB uses a relational model with column-based storage model.

Foreign Keys

Not Supported

Hardware Acceleration

Custom

SIMD optimised analytics.

Indexes

Hash Table

Indexing is available for symbol columns. The symbol datatype in QuestDB is used to store repeatitive strings. Internally, it is stored as a dictionary of integers and corresponding string values. The Index support for other types will be added over time.

Isolation Levels

Read Committed

QuestDB uses the READ COMMITTED isolation level.

Joins

Nested Loop Join Hash Join Semi Join

QuestDB supports INNER, LEFT OUTER, and CROSS join types. FULL joins are not yet implemented and are on our roadmap. In addition, QuestDB also supports ASOF, LT, and SPLICE join types particularly useful for time-series analytics.

Joins in QuestDB are internally implemented as nested loop joins and hash joins.

Logging

Physical Logging

Starting from version 7.0, QuestDB supported using WAL to ingest data. With the addition of WAL-supported tables, clients can write to disk independently without holding the lock of the table writer. Transaction numbers are provided by a central sequencer protecting a single source of truth. After applying the WALs, the data is applied to the table asynchronously by another job.

Parallel Execution

Intra-Operator (Horizontal)

Multi threaded SQL execution

Query Compilation

JIT Compilation

QuestDB uses a JIT compiler to improve performance by compiling the predicate evaluation part of the query. After determining whether a filter is suitable for compilation, the compiler frontend would transform the abstract syntax tree (AST) of the filter into an intermediate representation (IR). The IR is then processed by the compiler backend to emit vectorized machine code with the AVX2 instruction set.

Query Execution

Vectorized Model

QuestDB uses the vectorized execution model and takes advantage of SIMD operations for fast aggregations and predicate evaluation.

Query Interface

SQL

QuestDB has a SQL interface mostly compatible with the PostgreSQL dialect. It also extends the standard SQL with constructs simplifying the semantics of time-series queries (e.g. LATEST ON, SAMPLE BY). It supports a simple REST API, the Postgres wire protocol, as well as the InfluxDB line protocol (ILP). QuestDB now has an ILP client library in the following languages: .Net, C, C++, Go, Java, JavaScript, Python, and Rust. It is compatible with other third-party tools including Flink, Grafana, Kafka, MindsDB, Pandas, Prometheus, Redpanda, and Telegraf.

Storage Architecture

Disk-oriented

QuestDB uses a disk-oriented storage architecture with memory-mapped file data access.

Storage Model

Decomposition Storage Model (Columnar)

QuestDB uses a column-based storage model. Data is stored in tables with each column stored in its own file and its own native format. New data is appended to the bottom of each column to allow data to be organically retrieved in the same order that it was ingested.

Storage Organization

Log-structured

Stored Procedures

Not Supported

System Architecture

Shared-Nothing Embedded

Views

Not Supported