DeepDB (also sometimes called DeepSQL near the end of the project) was a proprietary MySQL storage engine designed for OLAP and OLTP workloads. Intended to scale MySQL to large scale data operations, it utilizes adaptive data structures and machine learning algorithms to optimize transactional workloads at big data scale. Different from classic B-Tree and LSM-Tree based storage engines, DeepDB is built on top of a new tree structure, called CASSI (Continuous Adaptive Sequential Summarization of Info), which dynamically configures the database during runtime to adapt to new hardware deployments. CASSI keeps running the three steps of analysis, adaption and optimization for high efficiency. Therefore, this storage engine allows enterprises to utilize MySQL without manual configuration under new hardware settings.[02][03][04]
- Source Code
- https://github.com/DeepFound/deep_engine[01]
- Developer
- Country of Origin
- US
- Start Year
- 2010
- End Year
- 2017 [10]
- Former Name
- DeepSQL
- Project Types
- Commercial, Open Source
- Written in
- C++
- Derived From
- MySQL
- Licenses
- AGPL v3, Proprietary
DeepDB (also sometimes called DeepSQL near the end of the project) was a proprietary MySQL storage engine designed for OLAP and OLTP workloads. Intended to scale MySQL to large scale data operations, it utilizes adaptive data structures and machine learning algorithms to optimize transactional workloads at big data scale. Different from classic B-Tree and LSM-Tree based storage engines, DeepDB is built on top of a new tree structure, called CASSI (Continuous Adaptive Sequential Summarization of Info), which dynamically configures the database during runtime to adapt to new hardware deployments. CASSI keeps running the three steps of analysis, adaption and optimization for high efficiency. Therefore, this storage engine allows enterprises to utilize MySQL without manual configuration under new hardware settings.[02][03][04]
History[05][06]
Deep Information Sciences was founded in 2010 based on research conducted at the University of New Hampshire. After the company went under in 2017, the source code of the DeepDB engine was released as open-source as part of a new Deep Software Foundation holding. A large portion of the source code of the system was a custom C++ implementation of the Java Development Kit software and not related to the DBMS itself.
Indexes[07][08]
DeepDB uses hyper-indexing and the new CASSI(Continuous Adaptive Sequential Summarization of Info) tree for indexing. CASSI tree is a persistent data structure that supports ACID transactions. It is improved from B-Trees in that they are able to collapse the internal structure through virtualizing and summarizing. Based on the the type of data and tasks, whether transactional, data stream capture, or analytics, the tree and indexes used in queries are dynamically adjusted to maximize hardware resources. For speed of lookup, sometimes it chooses to index every column in a database table.
Storage Architecture
DeepSQL turns MySQL into a cloud-ready, perpetually adapting database.
Storage Organization
A shadow copy is maintained for recovery on system crush. Disk snapshots are maintained to roll forward and back in data history.
System Architecture[02][09]
DeepDB storage engine is designed to replace MySQL's native storage engines, including InnoDB and MyISAM. It transforms MySQL into adaptive, self-tuning and highly performant database with full ACID compliance and brings additional machine-learning metrics to MySQL. The system is architected for complex environments and supports HTAP(Hybrid Transactional Processing).
Citations
10 sources- GitHub - DeepFound/deep_engine: High-performance C++ key/value database storage engine · GitHub github.com
- Deep Information Sciences Releases New DeepSQL Engine eweek.com
- http://misclassblog.com/databases-and-data-warehouses/deepsql-the-next-generation-of-database-optimization misclassblog.com
- DeepDB: General Purpose Database For Big Data Era | InformationWeek informationweek.com
- https://www.crunchbase.com/organization/deep-information-sciences-inc crunchbase.com
- https://www.businesswire.com/newsroom businesswire.com
- Cloud Computing recent news | InformationWeek informationweek.com
- deep_docs/Whitepaper_Continuous-Adaptive-Seq-Sum-Info.pdf at master · DeepFound/deep_docs · GitHub github.com
- https://www.businesswire.com/news/home/20160204005083/en/Deep-Data-Game-deepSQL-World’s-Cloud-Aware-Autonomic-Scaling businesswire.com
- https://finance.yahoo.com/news/deep-information-sciences-goes-open-130000559.html yahoo.com