History

The RisingLight project was initiated by Mingji Han, a PhD student at Boston University. The motivation is to build an OLAP database system that is simple enough to learn with modern programming technologies.

Checkpoints

RisingLight provides consistent snapshot by using merge-tree structure in the storage engine. The manifest file stores a full list of files of the database. Every write from users will produce a group of files called RowSet, and the information of the RowSet will be added to the manifest file.

Compression

The minimal encoding unit of RisingLight is data blocks, about 64KB in size, containing multiple cells in a table. RisingLight supports run-length block encoding.

Concurrency Control

RisingLight doesn't support transactions.

Data Model

RisingLight stores data in a relational way.

Foreign Keys

Indexes

In RisingLight, each table corresponds to a merge-tree on disk. Inside each merge-tree, there are multiple RowSets. Each RowSet is composed of multiple column files. Each column file contains multiple blocks, which is the minimal unit of the cache. RowSets can be sorted or unsorted. For sorted RowSets, the sparse index file will record the first cell of each block, so as to support efficient range scan.

Joins

RisingLight supports nested loop join and hash join. When there is at least one equal condition and the join type is inner join, RisingLight will use hash join. Otherwise, nested loop join will be used.

Logging

Parallel Execution

In RisingLight, different executors can be running on different threads.

Query Compilation

Query Execution[03]

RisingLight uses vectorized volcano model (aka. chunk at a time) for query execution. Furthermore, some arithmetic expressions have special SIMD implementation to make execution faster.

Query Interface[01]

RisingLight supports PostgreSQL as a query interface. It provides an interactive shell for users to issue SQL queries.

Storage Architecture[04]

RisingLight stores data on disk. Data are organized on disk using merge-tree structure. There are multiple RowSet directories inside the RisingLight database directory, where each RowSet belongs to a user table, organized in a merge-tree. Inside each RowSet directory, there are multiple column files and column index files.

Storage Model

A user table is cut horizontally as RowSets, and then vertically into column files. Data within one column are stored continuously in multiple column files.

Storage Organization

Stored Procedures

System Architecture

Views