AsterixDB is a DBMS that is highly parallel and, consequently, highly scalable. It has its own semi-structured data model similar to JSON/XML, as well as a custom query interface (SQL++) that incorporates much of SQL’s original functionality as well as new features. It partitions its data based on hashing and has a custom query execution engine, Apache Hyracks, to ensure that query plans take advantage of this partitioning and parallelize as much as possible.[04]
- Source Code
- https://github.com/apache/asterixdb[02]
- @AsterixDB
- Governance
- Apache Software Foundation
- Country of Origin
- US
- Start Year
- 2009 [19]
- Project Types
- Academic, Open Source
- Written in
- Java
- Operating System
- All OS with Java VM
- License
- Apache v2
AsterixDB is a DBMS that is highly parallel and, consequently, highly scalable. It has its own semi-structured data model similar to JSON/XML, as well as a custom query interface (SQL++) that incorporates much of SQL’s original functionality as well as new features. It partitions its data based on hashing and has a custom query execution engine, Apache Hyracks, to ensure that query plans take advantage of this partitioning and parallelize as much as possible.[04]
History[04]
Initially, AsterixDB was developed as a collaborative effort between students, faculty, and staff at UC Riverside and UC Irvine. Noting that at the time, there was no open-source parallel DBMS available to the public, the team aimed to fill that hole with AsterixDB. It was inspired by modern developments in parallel DBMSs, semi-structured data manipulation, and Hadoop, and the team aimed to combine the best of all three to form AsterixDB. After its initial open-source release in 2013, it was developed by the same team as before until it was accepted by the Apache Software Foundation in February 2015, where it began to be developed at the Foundation.
Compression[05]
AsterixDB has two options for compression: leaving data uncompressed, or using the Google Snappy naive compression algorithm, which focuses on maximizing speed while keeping a reasonable compression ratio. At the moment, only primary indexes can be compressed.
Concurrency Control[06][07]
Locks are only required on primary indexes; for secondary indexes, accesses do not require locks, but primary index lookups verify the integrity of secondary index lookups. Furthermore, locking is only done for lookups, inserts, and deletes -- other operations such as flushing to disk do not acquire locks. Notably, AsterixDB only supports single-statement transactions, and some more complicated SQL++ statements are even represented as multiple single-object statements and as such are represented as multiple transactions.
Data Model[08]
Asterix uses its own data representation, the Asterix Data Model (ADM), which is laid out similar to a JSON object. It currently supports common primitive types (booleans, strings, ints of different sizes, floats/dobules, binary) as well as geometric data types (point, line, rectangle, circle, polygon), and time-based data types (date, time, timestamp, interval, duration). It supports null/missing values, as well as derived types (objects, arrays, and multisets).
Indexes[03][09]
For primary indexes, AsterixDB uses Log-Structured Merge trees (LSM trees), and for secondary indexes, it allows B+trees, R trees, and inverted keyword indexes. However, these secondary indexes are “LSM-ified” to allow for LSM operations (inserting, searching, and merging components of the index) to be applied to them. This "LSM-ification" entails coupling the original in-memory secondary index with an in-memory B+tree called the deleted-key B+tree; while new updates and inserts go to the original in-memory index, deleted entries get recorded by storing their keys in the deleted-key B+tree.
Joins[10]
Currently, AsterixDB supports multiple join types (hash, nested loop, and broadcast), but it does not yet have effective statistics or selectivity estimates. Therefore, the default join algorithm is a hash-join. For non-equality predicates such as inequalities or “like” predicates for strings where, the default is a nested-loop join, since hashing can only be used for exact matches.
Logging[07]
AsterixDB uses logical logging at an index granularity, meaning each insert, delete, or update to an index generates a single log record. Logs have sequence numbers (LSNs) for the sake of the recovery phase. Unlike the ARIES protocol, where pages have page LSNs that indicate the most recent update applied to it, AsterixDB’s indexes have an index LSN, which indicates the most recent update applied to that particular index.
Query Interface[11][12][13]
Initially, AsterixDB used one custom query language, the Asterix Query Language (AQL). This is laid out like JSON but has more flexibility, making it a superset of JSON. However, the most recent documentation lists AQL as deprecated. The currently supported query interface for AsterixDB is SQL++, which is a superset of both SQL and JSON, making it more flexible than AQL.
Storage Architecture[14][15]
AsterixDB's primary LSM tree indexes have an in-memory component and a disk-based component, and data is flushed to disk when the in-memory component becomes too full.
Storage Model[09]
AsterixDB stores data in an NSM layout, hash-partitioning records to different nodes based on their primary keys.
Storage Organization[09]
AsterixDB uses a Log-Structured Merge tree (LSM tree) as its primary index. Secondary indexes are also "LSM-ified" to better fit with the system's overall storage organization.
Stored Procedures[16][17]
Users can define their own functions for AsterixDB in either SQL++ or Java.
System Architecture[18][09]
AsterixDB is a shared-nothing parallel DBMS that uses hash-based partitioning to split data among various nodes. Queries are routed to a single cluster controller. This then connects to node controllers and metadata node controllers, which in turn connect to individual nodes.
Views[13]
Neither SQL++ nor AQL have commands to create, delete, or access views, and as AsterixDB only supports those two languages, it has no support for views.
Citations
21 sources- Apache AsterixDB apache.org
- GitHub - apache/asterixdb: Mirror of Apache AsterixDB · GitHub github.com
- https://ci2.apache.org/projects/asterixdb/index.html apache.org
- About Apache AsterixDB apache.org
- Compression in AsterixDB - Apache AsterixDB - Apache Software Foundation apache.org
- https://ci.apache.org/projects/asterixdb/sqlpp/primer-sqlpp.html#Transaction_Support apache.org
- https://asterix.ics.uci.edu/pub/vldb14-storage.pdf#page=5 uci.edu
- https://ci.apache.org/projects/asterixdb/datamodel.html apache.org
- https://asterix.ics.uci.edu/pub/vldb14-storage.pdf uci.edu
- https://ci.apache.org/projects/asterixdb/sqlpp/primer-sqlpp.html#Query_2-B_-_Index_join apache.org
- https://ci.apache.org/projects/asterixdb/aql/manual.html apache.org
- http://forward.ucsd.edu/sqlpp.html ucsd.edu
- https://ci.apache.org/projects/asterixdb/sqlpp/manual.html apache.org
- https://asterix.ics.uci.edu/pub/vldb14-storage.pdf#page=2 uci.edu
- https://ci.apache.org/projects/asterixdb/ apache.org
- https://ci.apache.org/projects/asterixdb/udf.html apache.org
- https://ci.apache.org/projects/asterixdb/sqlpp/manual.html#Functions apache.org
- http://www.vldb.org/pvldb/vol7/p1905-alsubaiee.pdf#page=2 vldb.org
- ASTERIX uci.edu
- https://github.com/apache/asterixdb/commit/b248800041ec4174af5358c36803b21bb901d394 github.com
- https://github.com/apache/asterixdb/commit/25e84088375ddabc746bb8e9f54c3eb154767fb6 github.com