History[07][08][09][10][11]

BlazingSQL started as a GPU table joiner for multi-terabyte databases. The Aramburu brothers, Rodrigo and Felipe, founded a company in 2013 that provided analytical solutions and needed to speed up joins for pension fraud detection. It integrates with the open-source open GPU data science initiative, RAPIDS, which relies on NVIDIA GPUs.

The system is originally closed-source with a free community binary, but then became open-source in August 2019.

Checkpoints

It is unclear if BlazingSQL supports checkpointing.

Compression[12][13][14][15][16]

Historically, BlazingSQL supported compression and decompression on the GPU with bit-packing, delta encoding, dictionary encoding, and run-length encoding. This is currently disabled alongside its custom Simpatico file format. As of November 2018, it operates directly on Apache Parquet, CSV, and ORC. BlazingSQL does not currently write data and instead reads it from the data lake. It is able to operate directly on compressed data.

Concurrency Control[05]

BlazingSQL does not write data. It reads directly from the data lake, loading it into GPU data frames that can be shared with other BlazingSQL worker nodes through interprocess communication. Worker nodes do not have to be on the same machine, they can utilize different machines and different GPUs. BlazingSQL handles concurrency for the generation of result sets. However, the user is responsible for ensuring that the data in the data lake is internally consistent and free of corruption when it is queried.

Data Model[17]

BlazingSQL is a relational database. It accepts multiple in-memory formats (e.g. Apache Parquet) and provides a SQL interface for querying the data.

Foreign Keys

It is unclear if foreign keys are supported by BlazingSQL.

Hardware Acceleration[18][19][20][21]

BlazingSQL is hardware-accelerated with NVIDIA GPUs. Relevant columnar data is compressed, cached and sent to the GPU. The GPUs are used to speed up transforms, predicates, running predicates while skipping metadata, and to perform accelerated joins. This is accomplished by hooking into the cu* libraries that are part of the RAPIDS initiative, which are themselves bindings around NVIDIA's CUDA libraries.

Indexes

BlazingSQL does not appear to support indexes.

Isolation Levels[05]

BlazingSQL reads directly from immutable files.

Joins[22][23][24]

BlazingSQL supports transformations and hash joins (left, left-outer, full-outer) on all the column types supported by rapids.ai. Ordering, arithmetic, date transformations, predicates and group by operations are performed over vectors of data with GPU SIMD.

Logging[05]

BlazingSQL does not write data.

Query Compilation[25]

BlazingSQL uses RAPIDS libraries, which themselves use NVIDIA's CUDA. CUDA has support for JIT and code generation.

Query Execution[22]

BlazingSQL operations are vectorized on the GPU (SIMD).

Query Interface[12]

BlazingSQL exposes a Python connector for executing SQL commands.

Storage Architecture[05][06]

BlazingSQL caches the data which is read from the data lake. The cache is cascading, storing data in GPU memory, GPU memory, and finally SSD/NVME.

Storage Model[16][26]

BlazingSQL does not write data. It reads compressed data directly from the data lake and transmits relevant columns to the GPU. On the GPU, data is represented as a GPU DataFrame (GDF). GDFs are built on top of Apache Arrow, which is a columnar in-memory format.

Storage Organization[27]

BlazingSQL does not write data. It relies on external storage, i.e. data being available in a "data lake": huge quantities of raw data in some flat-file architecture, commonly Parquet.

Stored Procedures[23]

Unsupported.

System Architecture[28][29][30][31][32][20]

BlazingSQL can utilize multiple GPUs distributed across different servers. BlazingSQL also has a distributed cache. Upon reading from the data lake, data is cached on the worker nodes. If a worker node A requests data that was recently read from the data lake by another worker node B, worker node B is able to push the desired data to worker node A.

Views[33]

BlazingSQL supports both virtual and materialized views. Materialized views are currently not persistent.