AsterixDB

AsterixDB is a DBMS that is highly parallel and, consequently, highly scalable. It has its own semi-structured data model similar to JSON/XML, as well as a custom query interface (SQL++) that incorporates much of SQL’s original functionality as well as new features. It partitions its data based on hashing and has a custom query execution engine, Apache Hyracks, to ensure that query plans take advantage of this partitioning and parallelize as much as possible.

History

Initially, AsterixDB was developed as a collaborative effort between students, faculty, and staff at UC Riverside and UC Irvine. Noting that at the time, there was no open-source parallel DBMS available to the public, the team aimed to fill that hole with AsterixDB. It was inspired by modern developments in parallel DBMSs, semi-structured data manipulation, and Hadoop, and the team aimed to combine the best of all three to form AsterixDB. After its initial open-source release in 2013, it was developed by the same team as before until it was accepted by the Apache Software Foundation in February 2015, where it began to be developed at the Foundation.

Query Interface

Custom API

Initially, AsterixDB used one custom query language, the Asterix Query Language (AQL). This is laid out like JSON but has more flexibility, making it a superset of JSON. However, the most recent documentation lists AQL as deprecated. The currently supported query interface for AsterixDB is [SQL++](https://ci.apache.org/projects/asterixdb/sqlpp/manual.html), which is a superset of both SQL and JSON, making it more flexible than AQL.

Views

Not Supported

Neither SQL++ nor AQL have commands to create, delete, or access views, and as AsterixDB only supports those two languages, it has no support for views.

Concurrency Control

Two-Phase Locking (Deadlock Prevention) Two-Phase Locking (Deadlock Detection)

Locks are only required on primary indexes; for secondary indexes, accesses do not require locks, but primary index lookups verify the integrity of secondary index lookups. Furthermore, locking is only done for lookups, inserts, and deletes -- other operations such as flushing to disk do not acquire locks. Notably, AsterixDB only supports single-statement transactions, and some more complicated SQL++ statements are even represented as multiple single-object statements and as such are represented as multiple transactions.

Logging

Logical Logging

AsterixDB uses logical logging at an index granularity, meaning each insert, delete, or update to an index generates a single log record. Logs have sequence numbers (LSNs) for the sake of the recovery phase. Unlike the ARIES protocol, where pages have page LSNs that indicate the most recent update applied to it, AsterixDB’s indexes have an index LSN, which indicates the most recent update applied to that particular index.

Joins

Nested Loop Join Hash Join Broadcast Join Index Nested Loop Join

Currently, AsterixDB supports multiple join types (hash, nested loop, and broadcast), but it does not yet have effective statistics or selectivity estimates. Therefore, the default join algorithm is a hash-join. For non-equality predicates such as inequalities or “like” predicates for strings where, the default is a nested-loop join, since hashing can only be used for exact matches.

Storage Model

N-ary Storage Model (Row/Record)

AsterixDB stores data in an NSM layout, hash-partitioning records to different nodes based on their primary keys.

Stored Procedures

Supported

Users can define their own functions for AsterixDB in either SQL++ or Java.

Indexes

B+Tree R-Tree Inverted Index (Full Text) Log-Structured Merge Tree

For primary indexes, AsterixDB uses Log-Structured Merge trees (LSM trees), and for secondary indexes, it allows B+trees, R trees, and inverted keyword indexes. However, these secondary indexes are “LSM-ified” to allow for LSM operations (inserting, searching, and merging components of the index) to be applied to them. This "LSM-ification" entails coupling the original in-memory secondary index with an in-memory B+tree called the deleted-key B+tree; while new updates and inserts go to the original in-memory index, deleted entries get recorded by storing their keys in the deleted-key B+tree.

Storage Architecture

Disk-oriented

AsterixDB's primary LSM tree indexes have an in-memory component and a disk-based component, and data is flushed to disk when the in-memory component becomes too full.

Data Model

Document / XML

Asterix uses its own data representation, the Asterix Data Model (ADM), which is laid out similar to a JSON object. It currently supports common primitive types (booleans, strings, ints of different sizes, floats/dobules, binary) as well as geometric data types (point, line, rectangle, circle, polygon), and time-based data types (date, time, timestamp, interval, duration). It supports null/missing values, as well as derived types (objects, arrays, and multisets).

System Architecture

Shared-Nothing

AsterixDB is a shared-nothing parallel DBMS that uses hash-based partitioning to split data among various nodes. Queries are routed to a single cluster controller. This then connects to node controllers and metadata node controllers, which in turn connect to individual nodes.

Compression

Naïve (Record-Level)

AsterixDB has two options for compression: leaving data uncompressed, or using the Google Snappy naive compression algorithm, which focuses on maximizing speed while keeping a reasonable compression ratio. At the moment, only primary indexes can be compressed.

Storage Organization

Log-structured

AsterixDB uses a Log-Structured Merge tree (LSM tree) as its primary index. Secondary indexes are also "LSM-ified" to better fit with the system's overall storage organization.

AsterixDB Logo
Website

https://asterixdb.apache.org/

Source Code

https://github.com/apache/asterixdb

Tech Docs

https://ci.apache.org/projects/asterixdb/index.html

Developer

University of California-Irvine and University of California-Riverside

Country of Origin

US

Start Year

2009

Project Type

Academic, Open Source

Written in

Java

Operating Systems

All OS with Java VM

Licenses

Apache v2