Pebble

View Current Viewing Revision #6 from 11/24/2019 9:54 p.m.

Pebble is a key-value store written in Go and developed by Cockroach Labs. It is based on an unfinished Go port of LevelDB. Furthermore, additional features and optimizations are inspired by parts of RocksDB. Pebble is specifically created to suit the needs of CockroachDB, and has unique features that neither LevelDB nor RocksDB have.

History

The project started in 2018 as a replacement for RocksDB running inside of CockroachDB.

Indexes

Log-Structured Merge Tree

Pebble uses a LSM tree to store writes. All records are stored in batches, which in turn are stored in in-memory memtables and sstables. There is a separate skiplist implementation which contains offsets of the records within their batch in order to perform reads.

Storage Architecture

Disk-oriented

Data Model

Key/Value

Pebble uses two types of keys - user keys suited for the user API and 'Internal Keys' which are composed of the user key, a sequence number, and a value signifying the key type. The Internal Key type is used to support the needs of the LSM tree.