TileDB

View Current Viewing Revision #27 from 03/27/2023 8:18 p.m.

TileDB is a multi-dimensional array management system which uses an on-disk format that can store dense and sparse array data with support for fast updates. The current open-source version of TileDB functions as a storage layer, connecting user applications to storage backend.

History

TileDB was invented at the Intel Science and Technology Center for Big Data. The research center was a collaboration between Intel Labs and MIT. The research project was published in a VLDB 2017 paper. TileDB, Inc. was founded in February 2017 to further develop and maintain the DBMS.

Checkpoints

Not Supported

TileDB does not support checkpoints at the current stage.

Compression

Dictionary Encoding Run-Length Encoding

TileDB supports the following compressors: GZIP, Zstandard, LZ4, RLE, Bzip2, and Double-delta. Double-delta is a compressor created for TileDB, and is similar to Facebook's Gorilla system.

Concurrency Control

Not Supported

TileDB provides no transactional support in the current version. It only guarantees atomic reads and writes. TileDB allows users to build a transactional manager on top for concurrency control.

Data Model

Array / Matrix

TileDB uses a multi-dimensional array format that handles both sparse data and dense data. An array is composed of fragments, where each fragment is an array snapshot containing cells written in that write operation. Fragments can be categorised into sparse fragments and dense fragments. Sparse fragments store their elements in a global order. Dense fragments store their elements into regularised chunks in the index space.

Foreign Keys

Not Supported

TileDB does not support foreign keys at the current stage.

Isolation Levels

Read Uncommitted

TileDB provides no transaction support in the current version, and no isolation could be guaranteed.

Joins

Not Supported

TileDB does not support join operations at the current stage.

Logging

Not Supported

TileDB does not support logging at the current stage.

Query Interface

Custom API SQL

TileDB supports API for the following languages: SQL, C, C++, Python, Java, R, and Go. TileDB Python API is under further development, and subject to change.