Hyrise is a platform for research and education in the area of relational in-memory databases. The goal is to have a code base that is easy to understand and extend. This should make evaluations of new research concepts easier than it would be in full-featured DBMS products.
The initial version of Hyrise -- presented in PVLDB 2011 -- focussed on optimizing the table layout in order to optimize CPU caching for a given workload. It used a workload-aware highly flexible partitioning approach to cluster data. This flexible data layout came with a high number of virtual function calls/several indirections for data accesses. Due to the grown code base and a shifted research focus, a new version of Hyrise has been built from scratch starting in 2016. The new version sets its focus more on topics like NUMA support (cf. chunked partitioning), NVRAM, SQL optimization (cf. query optimizer), and Self-Driving.
Hyrise includes a compression framework based on C++ iterators and zero-cost abstractions. Such zero-cost abstractions avoid the runtime overheads of dynamic dispatching for increased compile times. As a heavy-weight compression algorithm, LZ4 is also supported.
For each row, three pieces of information are stored: The commit id of the transaction that successfully inserted the row (begin cid), that of the transaction that fully deleted it (end cid), and the transaction id of the transaction that currently modifies it. The linked Wiki page describes how these are used to calculate the row's visibility.
Adaptive Radix Trees (ARTs) and Group Key Indexes are used. These are created on a per-chunk basis and different chunks can have different types of indexes. Additionally, probabilistic filters are used for access avoidance in cases where no index exists.
Just-In-Time compilation is currently being developed; a first version is available and has to be specifically enabled. JIT operators are implemented in C++ code and can be executed as regular operators without a JIT compilation step. Code specialization (using LLVM) is used to inline function calls, replace constants, and perform other optimizations. The JIT compilation fuses multiple operators into one tight loop.
Most operators produce position lists that contain chunk ids and chunk offsets. Together with a pointer to the original table, these can be used to reference data without fully materializing it. The JIT engine (see above) uses a Tuple-at-a-Time Model.
The SQLPipeline is an interface that takes an SQL string and returns the result table(s). Hyrise uses its own SQL-Parser to translate SQL queries into an Abstract Syntax Tree (AST). These are converted into a logical query plan (LQP), which is optimized, translated into a physical query plan (PQP) and finally executed. The pipeline handles both query plan caching and stored procedures.
Hyrise started as one of the first databases with a hybrid memory layout. With the rewrite, only columnar storage is implemented. Hybrid layouts are planned to come back, but are not a high priority.
Stored Procedures are stored as named, static entries in the otherwise automatic query cache.
The current version of Hyrise runs on a single node. Replication, which was part of Hyrise1, is planned.
Views are stored as logical query plans (LQPs). For a query selecting from a view, the LQP is inserted at the position at which usually a stored table would appear before the optimizer is called. This makes it possible for the optimizer to optimize across view boundaries - for example pushing down additional predicates into the view.
Hasso Plattner Institute
Academic, Open Source
Linux, OS X