Kdb+

Kdb+ is a column-based relational time series database, developed by Kx Systems. Kdb+ database is designed to be used in financial area and store time series data.

History

In 1998, Kx Systems released **kdb**. Kx Systems then released **kdb+** as the 64-bit version in 2003. It is written in q language. Kdb+ is built to process large volume of time-series data in areas including finance and IoT.

Compression

Naïve (Record-Level)

Kdb+ supports on-disk compression with following algorithms: - kdb+ algorithm: default compression algorithm - gzip: supports different level of compression, larger compression rate needs more computation time - Google Snappy: time performance is better but compression rate is lower compared with previous two algorithms

Concurrency Control

Deterministic Concurrency Control

Kdb+ supports deterministic concurrency control.

Data Model

Relational

Kdb+ uses relational model.

Foreign Keys

Supported

Kdb+ supports referential integrity.

Indexes

B+Tree

Kdb+ supports both primary and secondary indexes.

Isolation Levels

Serializable

Kdb+ only supports SERIALIZABLE isolation level since it uses deterministic concurrency control.

Joins

Hash Join Semi Join

Kdb+ supports sql standard joins. It also supports as-of join and window join.

Logging

Physical Logging

Kdb+ uses physical logging and WAL. In-memory event-engine will log new data tolog file to ensure durability.

Query Execution

Vectorized Model

Kdb+ is written in q language and it's vector-based. Each function/operation in the query plan manipulates array/vector data.

Query Interface

SQL HTTP / REST

Kdb+ also supports ODBC/JDBC query interfaces.

Storage Architecture

Hybrid

Kdb+ has both in-memory and on-disk storage. New data is held in memory and old data is flushed to disk. The flush is controlled by event-engine. By default, event-engine will flush in-memory data to disk at daily basis. Rationale behind this design is the system wants to keep everyday new data in memory for fast query.

Storage Model

Decomposition Storage Model (Columnar)

Kdb+ uses DSM both for in-memory and on-disk storage.

Stored Procedures

Supported

Kdb+ supports user to write and store UDF in q language in addition to built-in functions.

System Architecture

Shared-Nothing

Kdb+ uses Lambda architecture on each single node. It has the following properties: - Data currently using stores in memory, while historical data is stored on disk. - New data come in from streaming sources. - Event-engine distribute data to downstream subscribers, including real-time database engine and streaming query engine. - Real-time database projects its content down to on-disk historical database for analytic use at daily basis, controlled by event-engine.

Kdb+ Logo
Website

https://kx.com/

Tech Docs

https://code.kx.com/

Former Name

kdb

Developer

Kx Systems

Country of Origin

US

Start Year

1998

Acquired By

First Derivatives plc

Project Type

Commercial

Supported languages

C, C#, C++, Go, Java, JavaScript, Lua, Matlab, Perl, PHP, Python, R, Scala

Operating Systems

Linux, OS X, Solaris, Windows

Licenses

Proprietary

Wikipedia

https://en.wikipedia.org/wiki/Kdb%2B