RisingWave

View Current Viewing Revision #5 from 04/30/2023 3:42 p.m.

Streaming

RisingWave is an open-source distributed streaming database targeting real-time analytics and event-driven applications. It uses the incremental computation model to process streaming data with low latency. RisingWave implements a traditional change propagation framework to keep user-defined materialized views up-to-date. An incremental checkpoint mechanism is used to ensure data consistency. It also has an elastic multi-node architecture with separate data and compute nodes. RisingWave is built from scratch with Rust, and is wire compatible with PostgreSQL.

History

RisingWave Database is built by RisingWave Labs (formerly known as Singularity Data), a database systems startup founded in 2021 by former IBM researcher and Amazon Redshift engineer Yingjun Wu.

While working at Amazon Redshift, Wu noticed that existing database systems cannot process streaming data efficiently and existing streaming systems were too complicated for most companies to use. This observation motivated Wu to found RisingWave Labs with a mission to “democratize stream processing”.

Checkpoints

Consistent

RisingWave uses the Chandy–Lamport algorithm to create consistent checkpoints.

To ensure that data is correct and consistent, read queries always fetch data from the most recent checkpoint. This means RisingWave does not ensure read-after-write consistency.

A local shared buffer is used to stage uncommitted write batches submitted by operators. The storage manager will notify all operators to commit their buffered writes into the shared storage when checkpoint trigger message has reached all operators.