AresDB

View Current Viewing Revision #5 from 11/25/2019 12:02 a.m.

AresDB is a GPU-based real-time analytics storage and query engine with low memory overhead, real-time upserts with primary key deduplication, and time series aggregations on both streaming and finite dimensional data.

History

Developed by Uber to meet their specific need "to make similar queries over relatively small, yet highly valuable, subsets of data (with maximum data freshness) at high QPS and low latency," with queries such as time series aggregations over geofences.

Logging

Logical Logging

Log files contain description of database upserts which must be replayed to rebuild the database after a crash.

Parallel Execution

Intra-Operator (Horizontal)

Executes queries with the one operation per kernel (OOPK) model.

Storage Model

Decomposition Storage Model (Columnar)

Query Execution

Vectorized Model

AresDB works with vector batches that are efficiently processed in parallel using the Thrust library.

Query Interface

Custom API

Uses a proprietary execution language called Ares Query Language (AQL) which is based in the JSON format.

Data within the archival delay of a table is kept uncompressed in live batches, while everything else is stored in compressed archival batches. If new data is ingested that is outside the archival array, it's added to an archival backfill queue which will be inserted into the archived batches asynchronously.

Revision #5 | Updated 11/25/2019 12:02 a.m.