Drill is a database system designed for Big Data exploration. It is an open-source, distributed SQL query system based on Google's Dremel query system, and it features a columnar execution engine. Drill is the only distributed SQL engine in the world that does not require schemas. It was designed from the ground up, and it supports many NoSQL databases and file systems, with the ability for a single query to join data from multiple datastores.[05][06][04][01]
- Website
- https://drill.apache.org[01]
- Source Code
- https://github.com/apache/drill[02]
- Tech Docs
- https://drill.apache.org/docs/[03]
- Developer
- Country of Origin
- US
- Start Year
- 2012 [11]
- Project Type
- Open Source
- Written in
- Java
- Supported Languages
- SQL
- Inspired By
- BigQuery
- License
- Apache v2
Drill is a database system designed for Big Data exploration. It is an open-source, distributed SQL query system based on Google's Dremel query system, and it features a columnar execution engine. Drill is the only distributed SQL engine in the world that does not require schemas. It was designed from the ground up, and it supports many NoSQL databases and file systems, with the ability for a single query to join data from multiple datastores.[05][06][04][01]
History[07][08][04]
In 2010, Google published a paper titled "Dremel: Interactive Analysis of Web-Scale Datasets" that described a scalable database system designed for "interactive analysis of nested data". The Dremel system is available today under Google's BigQuery system. Development of Apache Drill began in 2012, with the goal of replicating the capabilities of Dremel. Initial goals of the system included support for multiple storage systems, file formats, query languages, and data sources, as well as the ability to scale over 10,000 servers and process petabytes of data in seconds.
Checkpoints[09]
Drill adopts optimistic query execution, which assumes that failures occur rarely during queries. Therefore, it does not take checkpoints. With its pipelined query execution model, single queries are simply reran when they fail.
Concurrency Control[10]
Drill supports Optimistic Concurrency Control. It plans queries in fragments, assuming that all of the fragments can be completed in parallel. Larger fragments are broken into smaller fragments, which are run in clusters until the whole fragment is complete.
Data Model
Drill features a JSON self-describing data model that supports language independence and loosely defined, weak data typing.
Citations
11 sources- https://drill.apache.org apache.org
- GitHub - apache/drill: Apache Drill is a distributed MPP query layer for self describing data · GitHub github.com
- Documentation - Apache Drill apache.org
- Apache Drill - Wikipedia wikipedia.org
- Drill Introduction - Apache Drill apache.org
- https://mapr.com/products/apache-drill mapr.com
- DrillProposal - INCUBATOR - Apache Software Foundation apache.org
- The Apache Software Foundation Announces Apache™ Drill™ as a Top-Level Project - The ASF Blog apache.org
- Architecture - Apache Drill apache.org
- Drill Query Execution - Apache Drill apache.org
- First commit github.com